Sparse Random Networks for Communication-Efficient Federated Learning
Authors: Berivan Isik, Francesco Pase, Deniz Gunduz, Tsachy Weissman, Zorzi Michele
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show improvements in accuracy, communication (less than 1 bit per parameter (bpp)), convergence speed, and final model size (less than 1 bpp) over relevant baselines on MNIST, EMNIST, CIFAR10, and CIFAR-100 datasets, in the low bitrate regime. In this section, we empirically show the performance of Fed PM in terms of accuracy, bitrate, converge speed, and the final model size. |
| Researcher Affiliation | Academia | Stanford University, University of Padova, Imperial College London berivan.isik@stanford.edu, pasefrance@dei.unipd.it |
| Pseudocode | Yes | We give the pseudocode for Fed PM in Appendix A. Algorithm 1 Fed PM. Algorithm 2 Bayes Agg. |
| Open Source Code | Yes | The codebase for this work is open-sourced at https://github.com/BerivanIsik/sparse-random-networks. |
| Open Datasets | Yes | We consider four datasets: CIFAR-10 with 10 classes, CIFAR-100 (Krizhevsky et al., 2009) with 100 classes, MNIST (Deng, 2012) with 10 classes, and EMNIST (Cohen et al., 2017) with 47 classes. We only used publicly available standard datasets and included links to them in the manuscript. |
| Dataset Splits | Yes | We demonstrate the efficacy of our strategy on MNIST, EMNSIT, CIFAR-10, and CIFAR-100 datasets under both IID and non-IID data splits; and show improvements in accuracy, bitrate, convergence speed, and final model size over relevant baselines, under various system configurations. Clients perform 3 local epochs in all experiments. We provide additional details on the experimental setup in Appendix D. Clients performed 3 local epochs with a batch size of 128 and a local learning rate of 0.1 in all the experiments. |
| Hardware Specification | Yes | We conducted our experiments on NVIDIA Titan X GPUs on an internal cluster server, using 1 GPU per one run. |
| Software Dependencies | No | While the paper states that its codebase is open-sourced ('The codebase for this work is open-sourced at https://github.com/BerivanIsik/sparse-random-networks.'), it does not explicitly list any specific software dependencies (e.g., Python 3.x, PyTorch 1.x) with their version numbers. |
| Experiment Setup | Yes | Clients perform 3 local epochs in all experiments. We provide additional details on the experimental setup in Appendix D. Clients performed 3 local epochs with a batch size of 128 and a local learning rate of 0.1 in all the experiments. |