Fast Composite Optimization and Statistical Recovery in Federated Learning
Authors: Yajie Bao, Michael Crawshaw, Shan Luo, Mingrui Liu
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical experiments in both synthetic and real data demonstrate that our methods perform better than other baselines. |
| Researcher Affiliation | Academia | Yajie Bao 1 Michael Crawshaw 2 Shan Luo 1 Mingrui Liu 2 1School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China 2Department of Computer Science, George Mason University, Fairfax, VA 22030, USA. Correspondence to: Mingrui Liu <mingruil@gmu.edu>. |
| Pseudocode | Yes | Algorithm 1 Fast-Fed DA(w0, R, E, µ, L), Algorithm 2 C-Fed DA(w0, R, E, ϵ0, µ, L, λ), Algorithm 3 Multi-stage C-Fed DA |
| Open Source Code | No | The paper mentions computing resources (URL: https://orc.gmu.edu) but does not provide any explicit statement about releasing its own source code or a link to a code repository for the methodology described. |
| Open Datasets | Yes | Federated EMNIST (Caldas et al., 2019) dataset of handwritten letters and digits. We train a multi-class logisitc regression model on two versions of this dataset: EMNIST-10 (digits only, 10 classes), and EMNIST-62 (all alphanumeric characters, 62 classes). Following (Yuan et al., 2021), we use only 10% of the samples, which is sufficient to train a logistic regression model. Our subsampled EMNIST-10 dataset consists of 367 clients with an average of 99 examples each, while EMNIST-62 consists of 379 clients with an average of 194 examples each. |
| Dataset Splits | No | The paper mentions training on EMNIST datasets and evaluating 'Train Accuracy' and 'Test Accuracy' in its figures, but it does not specify a 'validation' dataset split or provide details on how the data was partitioned for training, validation, and testing. |
| Hardware Specification | No | Computations were run on ARGO, a research computing cluster provided by the Office of Research Computing at George Mason University (URL: https://orc.gmu.edu). |
| Software Dependencies | No | The paper describes the algorithms and experiments but does not list specific software dependencies with version numbers. |
| Experiment Setup | Yes | For federated sparse linear regression... the batch size is 10 and the regularization parameter is λ = 0.55. The hyperparameters for Fast-Fed DA are µ = 0.1 and L = 550. For C-Fed DA and MC-Fed DA, we choose µ = 0.1 and L = 600. For Fed DA and Fed Mi D, we set the server learning rate ηs = 1.0 and tuned the client learning rate ηc by selecting the best performing value over the set {0.0001, 0.001, 0.01, 0.1}, which was 0.001 for both baselines. ...For both experiments, we use a batch size of 25, a regularization parameter λ = 10 4, and we sample 36 clients to perform local updates at each communication round. For EMNIST-10, each sampled client performs K = 40 updates per communication round for R = 15000 rounds. For EMNIST-62, K = 10 and R = 75000. |