Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient Federated Learning
Authors: Mohammad Mahdi Rahimi, Hasnain Irshad Bhatti, Younghyun Park, Humaira Kousar, Jaekyun Moon
NeurIPS 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our algorithm s effectiveness is assessed on three image classification datasets: FMNIST [52], MNIST [53], and CIFAR-10 [54]. ... The accuracy of Evo Fed, compared with multiple baseline methods and different datasets, is shown in Fig. 8 (a), (b), and (c). ... Table 1 summarizes the performance of different schemes on MNIST, FMNIST, and CIFAR-10 datasets, focusing on communication cost and accuracy. |
| Researcher Affiliation | Academia | Mohammad Mahdi Rahimi Hasnain Irshad Bhatti Younghyun Park Humaira Kousar Jaekyun Moon KAIST EMAIL EMAIL |
| Pseudocode | Yes | Algorithm 1 Evo Fed: Federated Learning with Evolutionary Strategies |
| Open Source Code | No | The paper states: 'Our Evo Fed framework is built using JAX [55], which facilitates extensive parallelization and, in particular, consistent random number generation across a large number of nodes. We have implemented our framework on the Evosax [56] library, a convenient tool for the ES algorithm.' These are external libraries, not the authors' own code for Evo Fed. There is no explicit statement about releasing the source code for the described methodology or a direct link to a repository containing it. 'Supplementary Materials' are mentioned, but not specified to contain the code. |
| Open Datasets | Yes | Our algorithm s effectiveness is assessed on three image classification datasets: FMNIST [52], MNIST [53], and CIFAR-10 [54]. Both MNIST and FMNIST contain 60,000 training samples and 10,000 test samples, whereas CIFAR-10 is composed of 50,000 training samples and 10,000 test samples. |
| Dataset Splits | Yes | Both MNIST and FMNIST contain 60,000 training samples and 10,000 test samples, whereas CIFAR-10 is composed of 50,000 training samples and 10,000 test samples. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running its experiments. It mentions 'increased local processing loads' and 'client computational load' in general terms. |
| Software Dependencies | No | The paper states: 'Our Evo Fed framework is built using JAX [55]... We have implemented our framework on the Evosax [56] library...' While the software names are mentioned, specific version numbers for these libraries (e.g., JAX 0.3.14, Evosax 0.0.1) are not provided in the text or their references. |
| Experiment Setup | Yes | Evo Fed is configured with a population size of 128 and a mini-batch size of 256 for MNIST / FMNIST and 64 for CIFAR-10. We perform ten local epochs (performing ten BP steps before fitness calculation) and train over 1,000 global rounds. |