Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Revisiting Sparsity Hunting in Federated Learning: Why does Sparsity Consensus Matter?
Authors: Sara Babakniya, Souvik Kundu, Saurav Prakash, Yue Niu, Salman Avestimehr
TMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental analysis on diverse models and datasets shows the superiority of FLASH in closing the gap with an unpruned baseline while yielding up to 10.1% improved accuracy with 10.26 fewer communication, compared to existing alternatives, at similar hyperparameter settings. [...] We conduct experiments on MNIST, FEMNIST, CIFAR-10, CIFAR-100, and Tiny Image Net with different models for both IID and non-IID client data partitioning. |
| Researcher Affiliation | Collaboration | Sara Babakniya EMAIL Department of Computer Science University of Southern California, USA Souvik Kundu EMAIL Intel Labs, San Diego, USA Saurav Prakash EMAIL Department of Electrical and Computer Engineering University of Southern California, USA Yue Niu EMAIL Department of Electrical and Computer Engineering University of Southern California, USA Salman Avestimehr EMAIL Department of Electrical and Computer Engineering University of Southern California, USA |
| Pseudocode | Yes | Algorithm 1: FLASH Training. [...] Algorithm 2: Hetero-FLASH Training. |
| Open Source Code | Yes | Code is available at https://github.com/Sara Babak N/flash_fl.git |
| Open Datasets | Yes | We conduct experiments on MNIST, FEMNIST, CIFAR-10, CIFAR-100, and Tiny Image Net with different models for both IID and non-IID client data partitioning. [...] MNIST (Le Cun & Cortes, 2010) on MNISTNet (Mc Mahan et al., 2017), FEMNIST on model described in (Caldas et al., 2018) and CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009) and Tiny Image Net (Pouransari & Ghili, 2014) on Res Net18 |
| Dataset Splits | Yes | For data partitioning of MNIST, CIFAR-10, CIFAR-100, and Tiny Image Net, we use Latent Dirichlet Allocation (LDA)(Reddi et al., 2020) with three different α (α = 1000, 1, 0.1). In this allocation, decreasing the value of the alpha increases the data heterogeneity among the clients. FEMNIST is built from handwriting 3400 different users (Han et al., 2020), making it inherently non-IID. [...] Figure 6: Test accuracy vs. round for CIFAR-10 with (a) d = 0.05 (b) d = 0.1. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments are provided in the paper. |
| Software Dependencies | No | The paper does not explicitly state specific software dependencies with version numbers, such as Python or PyTorch versions. |
| Experiment Setup | Yes | Training hyperparameters. We use starting learning rate (ηinit) as 0.1 which exponentially decayed to 0.001 (ηend). Specifically, learning rate for participants at round t is ηt = ηinit(exp( t T log( ηinit ηend ))). In all the sparse learning experiments, the pruning rate is 0.254. Other training hyperparameters can be found in Table 1. Furthermore, all the results are averaged over three different seeds. [...] Table 1: FL training settings considered in this work. Dataset Model Data partitioning Rounds (T) Clients (CN) Clients/Round (cr, cd) Optimizer Aggregation Local epoch (E) Batch size |