SpaFL: Communication-Efficient Federated Learning With Sparse Models And Low Computational Overhead
Authors: Minsu Kim, Walid Saad, Merouane DEBBAH, Choong Hong
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that Spa FL improves accuracy while requiring much less communication and computing resources compared to sparse baselines. |
| Researcher Affiliation | Academia | Minsu Kim Virginia Tech Walid Saad Virginia Tech Merouane Debbah Khalifa University Choong Seon Hong Kyung Hee University |
| Pseudocode | Yes | The overall algorithm is illustrated in Fig 1. and summarized in Algorithm 1. |
| Open Source Code | Yes | The code is available at https://github.com/news-vt/Spa FL_Neru IPS_2024 |
| Open Datasets | Yes | We conduct experiments on three image classification datasets: FMNIST [37], CIFAR-10, and CIFAR-100 [38] datasets with NVIDA A100 GPUs. To distribute datasets in a non-iid fashion, we use Dirichlet (0.2) for FMNIST and Dirichlet (0.1) for CIFAR-10 and CIFAR-100 datasets as done in [39] with N = 100 clients. |
| Dataset Splits | No | The paper specifies training data distribution and explicitly mentions a 'test dataset' for evaluation, but it does not specify a separate 'validation' dataset split. |
| Hardware Specification | Yes | We conduct experiments on three image classification datasets: FMNIST [37], CIFAR-10, and CIFAR-100 [38] datasets with NVIDA A100 GPUs. |
| Software Dependencies | No | The paper mentions using 'Py Torch' in the appendix, but it does not specify a version number for PyTorch or any other software library. |
| Experiment Setup | Yes | For FMNIST dataset, we use the Lenet-5-Caffe. For the Lenet model, we set η(t) = 0.001, E = 5, α = 0.002, and a batch size to be 64. For CIFAR-10 dataset, we use a convolutional neural network (CNN) model with seven layers used in [40] with η(t) = 0.01, E = 5, α = 0.00015, and a batch size of 16. We adopt the Res Net-18 model for CIFAR-100 dataset with η(t) = 0.01, E = 7, α = 0.0007, and a batch size of 64. The learning rate of CIFAR-100 is decayed by 0.993 at each communication round. |