Revisiting Ensembling in One-Shot Federated Learning
Authors: Youssef Allouah, Akash Dhasade, Rachid Guerraoui, Nirupam Gupta, Anne-marie Kermarrec, Rafael Pinot, Rafael Pires, Rishi Sharma
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We showcase the effectiveness of FENS through exhaustive experiments spanning several datasets and heterogeneity levels. Through extensive evaluations on several benchmark datasets (CIFAR100, CIFAR-10, SVHN, and AG-News) across different heterogeneity levels, we demonstrate the efficacy of FENS in achieving FL-like accuracy at OFL-like communication cost. |
| Researcher Affiliation | Academia | 1EPFL 2University of Copenhagen 3Sorbonne Université and Université Paris Cité, CNRS, LPSM |
| Pseudocode | Yes | Algorithms 1 and 2 (Appendix C) provide the pseudo for FENS. |
| Open Source Code | Yes | Source code available at: https://github.com/sacs-epfl/fens. |
| Open Datasets | Yes | We consider three standard vision datasets with varying level of difficulty, including SVHN [30], CIFAR-10 [30] and CIFAR-100 [30], commonly used in several OFL works [7, 10, 46] as well as one language dataset AG-News [47]. For our experiments involving the realistic healthcare FLamby benchmark, we experiment with 3 datasets: Fed-Camelyon16, Fed-Heart-Disease, and Fed-ISIC2019. |
| Dataset Splits | Yes | The testing set of each dataset is split (50-50%) for validation and testing. We use the validation set to tune hyperparameters and always report the accuracy on the testing split. In FENS, each client performs local training using 90% of their local training data while reserving 10% for the iterative aggregator training. |
| Hardware Specification | Yes | We use a cluster comprising a mix of 2x Intel Xeon Gold 6240 @ 2.5 GHz of 36 hyper-threaded cores and 2x AMD EPYC 7302 @ 3 GHz of 64 hyper-threaded cores, equipped with 4x NVIDIA Tesla V100 32G and 8x NVIDIA Tesla A100 40G GPU respectively. |
| Software Dependencies | No | We use the standard Py Torch quantization library to implement quantization in FENS. The paper mentions "Py Torch quantization library" but no specific version number for it or other software dependencies. |
| Experiment Setup | Yes | For the CIFAR-10 and CIFAR-100 datasets, each client conducts local training for 500 epochs utilizing SGD as the local optimizer with an initial learning rate of 0.0025. For the SVHN and AG-News datasets, local training extends to 50 and 20 epochs respectively with a learning rate of 0.01. The learning rate is decayed using Cosine Annealing across all datasets. We tune learning rates for each algorithm. (Table 6) |