Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learning
Authors: Huancheng Chen, Haris Vikalo
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experimental results demonstrate that in non-IID settings Hi CS-FL achieves faster convergence than state-of-the-art FL client selection schemes. |
| Researcher Affiliation | Academia | Huancheng Chen University of Texas at Austin huanchengch@utexas.edu Haris Vikalo University of Texas at Austin hvikalo@ece.utexas.edu |
| Pseudocode | Yes | Algorithm 1 Hi CS-FL |
| Open Source Code | No | The paper does not explicitly state that its code is released or provide a link to a code repository. |
| Open Datasets | Yes | We evaluate the proposed Hi CS-FL algorithm on four benchmark datasets (FMNIST, CIFAR10, Mini-Image Net and THUC news) using different model architectures. ... The experimental results were obtained using Pytorch [22]. In the experiments involving Mini-Image Net and THUC news, each client fine-tuned a pretrained Res Net18 [13] and learned a Text RNNs [20], respectively. To generate non-IID data partitions, we follow the strategy in [35], utilizing Dirichlet distribution... |
| Dataset Splits | No | The paper describes how data is partitioned across clients using Dirichlet distribution for non-IID settings, and mentions 'training loss' and 'test accuracy', but does not provide specific percentages or counts for training, validation, and test dataset splits. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, memory, or specific cloud instance types used for the experiments. |
| Software Dependencies | No | The paper states 'The experimental results were obtained using Pytorch [22]' and mentions 'Adam [15]' as an optimizer, but does not provide specific version numbers for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | The learning rate was initially set to 0.001 and then decreased every 10 iterations, with a decay factor 0.5. The number of global communication rounds was set to 200, 500, 100 and 100 for the experiments on FMNIST, CIFAR10, Mini-Image Net and THUC news, respectively. In all the experiments, the number of local epochs R was set to 2 and the size of a mini-batch was set to 64. The sampling rate (fraction of the clients participating in a training round) was set to 0.1 for the experiments on FMNIST/THUC news, and to 0.2 for the experiments on CIFAR10/Mini-Image Net. ... In all experiments, the hyper-parameter µ of the regularization term in Fed Prox [19] was set to 0.1. ... For Hi CS-FL (our method), the scaling parameter T (temperature) used in data heterogeneity estimation was set to 0.0025 in the experiments on FMNIST and to 0.0015 in the experiments on CIFAR10/Mini-Image Net. In all experiments, parameter λ which multiplies the difference between clients estimated data heterogeneity (used in clustering) was set to 10. ... The coefficient γ0 was set to 4 in the experiments on FMNIST and CIFAR10 while set to 2 in the experiments on Mini-Image Net. |