An Efficient Framework for Clustered Federated Learning
Authors: Avishek Ghosh, Jichan Chung, Dong Yin, Kannan Ramchandran
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In the experiments, we show that our algorithm can succeed even if we relax the requirements on initialization with random initialization and multiple restarts. We also present experimental results showing that our algorithm is efficient in non-convex problems such as neural networks. |
| Researcher Affiliation | Collaboration | Avishek Ghosh Dept of EECS, UC Berkeley Berkeley, CA 94720 avishek_ghosh@berkeley.edu Jichan Chung* Dept of EECS, UC Berkeley Berkeley, CA 94720 jichan3751@berkeley.edu Deep Mind Mountain View, CA 94043 dongyin@google.com Kannan Ramchandran Dept of EECS, UC Berkeley Berkeley, CA 94720 kannanr@berkeley.edu |
| Pseudocode | Yes | The algorithm is formally presented in Algorithm 1 and illustrated in Figure 1. |
| Open Source Code | Yes | Implementation of our experiments is open sourced at https://github.com/jichan3751/ifca. |
| Open Datasets | Yes | We also create clustered FL datasets based on the MNIST [19] and CIFAR-10 [18] datasets. For Rotated MNIST, recall that the MNIST dataset has 60000 training images and 10000 test images with 10 classes. We provide additional experimental results on the Federated EMNIST (FEMNIST) [2], which is a realistic FL dataset where the data points on every worker machine are the handwritten digits or letters from a specific writer. |
| Dataset Splits | No | The paper does not provide explicit information about a validation dataset split. |
| Hardware Specification | No | The paper does not specify the exact hardware used for running the experiments (e.g., GPU models, CPU types, or memory). |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | We then run Algorithm 1 for 300 iterations, with a constant step size. For k = 2 and k = 4, we choose the step size in {0.01, 0.1, 1}, {0.5, 1.0, 2.0}, respectively. For Local Update step in Algorithm 1, we choose = 10 and step size γ = 0.1. For CIFAR experiments, we choose |Mt| = 0.1m, and apply step size decay 0.99, and we also set = 5 and batch size 50 for Local Update process, following prior works [28]. |