Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients
Authors: Aritra Mitra, Rayana Jaafar, George J. Pappas, Hamed Hassani
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we provide numerical results for Fed Lin on a least squares problem to validate our theory. In Appendix K, we also provide additional numerical results on a logistic regression problem. |
| Researcher Affiliation | Academia | Department of Electrical and Systems Engineering {amitra20,rayanaj,pappasg,hassani}@seas.upenn.edu |
| Pseudocode | Yes | Algorithm 1 Fed Lin |
| Open Source Code | No | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [No] |
| Open Datasets | No | To generate synthetic data, for each client i S = {1, . . . , 20}, we generate Ai and bi according to the model bi = Aixi + εi, where xi is a weight vector and εi R500 is a disturbance. In particular, we generate [Ai]jk i.i.d. N(0, 1), and εi N(0, 0.5I500), i S. |
| Dataset Splits | No | The paper describes generating synthetic data and states it will focus on a deterministic setting for experiments, but it does not specify any training, validation, or test dataset splits. |
| Hardware Specification | No | Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No] |
| Software Dependencies | No | The paper describes specific hyperparameters (e.g., step size η), but it does not provide details on specific software dependencies or their version numbers (e.g., libraries, frameworks, or solvers). |
| Experiment Setup | Yes | The constant η is fixed at 10^−2. ... The constant η is fixed at 5 × 10^−4. ... We set the number of local steps H = 20, the statistical heterogeneity parameter α = 10, and use a step-size of 10^−3 for both algorithms (the step-size was tuned to get best results). |