Resolving the Tug-of-War: A Separation of Communication and Learning in Federated Learning
Authors: Junyi Li, Heng Huang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical validation shows the superior performance of Fed Sep over various baselines in these tasks. In this section, we perform numerical experiments to validate the efficacy of Fed Sep in solving communication-efficient FL and model-heterogeneous FL. |
| Researcher Affiliation | Academia | Junyi Li Computer Science University of Maryland College Park junyili.ai@gmail.com Heng Huang Computer Science University of Maryland College Park henghuanghh@gmail.com |
| Pseudocode | Yes | Algorithm 1 Separating Communication and Learning in FL (Fed Sep) |
| Open Source Code | No | The paper mentions 'The code is written with Pytorch [39]' but does not provide a link or explicit statement about releasing its own source code for the described methodology. |
| Open Datasets | Yes | Dataset. We consider MNIST [29] and CIFAR-10 [27] in our experiments. |
| Dataset Splits | No | The paper uses standard datasets (MNIST, CIFAR-10, CIFAR-100) but does not explicitly provide specific percentages, counts, or citations for the training, validation, and test dataset splits, only mentioning data distribution among clients and that 'For Fed Sep, the validation and training set are the same' for a specific internal process. |
| Hardware Specification | Yes | We used servers with AMD EPYC 7763 64-core CPU and 8 NVIDIA V100 GPUs to run our experiments. |
| Software Dependencies | No | The paper states 'The code is written with Pytorch [39], and the Federated Learning environment is simulated via Pytorch.Distributed Package.' but does not provide specific version numbers for PyTorch or other software dependencies. |
| Experiment Setup | Yes | The local learning steps are set as I = 5. (Figure 2 caption). In experiments, the size of the submodel p (Eq. (6)) of a client is randomly chosen from {1, 0.5, 0.25, 0.125, 0.0625} (Section 5.2). Also states 'Please refer to the Appendix for the hyper-parameter selection of Fed Sep and baselines.' |