FedImpro: Measuring and Improving Client Update in Federated Learning
Authors: Zhenheng Tang, Yonggang Zhang, Shaohuai Shi, Xinmei Tian, Tongliang Liu, Bo Han, Xiaowen Chu
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that Fed Impro can help FL defend against data heterogeneity and enhance the generalization performance of the model. 5 EXPERIMENTS |
| Researcher Affiliation | Academia | 1 Department of Computer Science, Hong Kong Baptist University 2 Harbin Institute of Technology, Shenzhen 3 University of Science and Technology of China 4 Sydney AI Centre, The University of Sydney 5 DSA Thrust, The Hong Kong University of Science and Technology (Guangzhou) |
| Pseudocode | Yes | Algorithm 1 Framework of Fed Impro. |
| Open Source Code | Yes | To ensure the reproducibility of experimental results, we open source our code at https://github.com/wizard1203/FedImpro. |
| Open Datasets | Yes | We verify Fed Impro with four datasets commonly used in the FL community, i.e., CIFAR-10 (Krizhevsky & Hinton, 2009), FMNIST (Xiao et al., 2017), SVHN (Netzer et al., 2011), and CIFAR-100 (Krizhevsky & Hinton, 2009). |
| Dataset Splits | No | We use the Latent Dirichlet Sampling (LDA) partition method to simulate the Non-IID data distribution, which is the most used partition method in FL (He et al., 2020b; Li et al., 2021c; Luo et al., 2021). The paper describes how data is partitioned (LDA) and mentions training and testing samples (e.g., for FEMNIST), but does not explicitly provide percentages or counts for training, validation, and test splits across all experiments, nor does it specify a general validation split methodology. |
| Hardware Specification | Yes | We conduct experiments using GPU GTX-2080 Ti, CPU Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz. The operating system is Ubuntu 16.04.6 LTS. |
| Software Dependencies | Yes | The Pytorch version is 1.8.1. The Cuda version is 10.2. |
| Experiment Setup | Yes | We conduct experiments with two different Non-IID degrees, a = 0.1 and a = 0.05. We simulate cross-silo FL with M = 10 and cross-device FL with M = 100. To simulate the partial participation in each round, the number of sample clients is 5 for M = 10 and 10 for M = 100. And for all experiments, we use SGD as optimizer for all experiments, with batch size of 128 and weight decay of 0.0001. |