FedScale: Benchmarking Model and System Performance of Federated Learning at Scale
Authors: Fan Lai, Yinwei Dai, Sanjay Singapuram, Jiachen Liu, Xiangfeng Zhu, Harsha Madhyastha, Mosharaf Chowdhury
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We combine the two to perform systematic benchmarking experiments and highlight potential opportunities for heterogeneity-aware co-optimizations in FL. |
| Researcher Affiliation | Academia | 1Department of Computer Science, University of Michigan 2Department of Computer Science, University of Washington. |
| Pseudocode | Yes | Figure 9 shows a code snippet for adding plugins by inheritance, demonstrating structured steps for a method: "from fedscale.core.client import Client class Customized_Client(Client): def train(self,client_data,model,conf): ... return training_result" |
| Open Source Code | Yes | Fed Scale is open-source and actively maintained by contributors from different institutions at http://fedscale.ai. |
| Open Datasets | Yes | Fed Scale currently has 20 realistic FL datasets (Table 2) across diverse practical FL scenarios and disciplines." and "Google Speech Commands. A speech recognition dataset (Warden, 2018) |
| Dataset Splits | Yes | Following the practical FL deployments (Yang et al., 2018), we assign the clients of each dataset into the training, validation and testing groups, to get its training, validation and testing set. |
| Hardware Specification | Yes | We use 10 NVIDIA Tesla P100 GPUs in our evaluations. Each GPU node has a P100 GPU with 12GB GPU memory and 192GB RAM. |
| Software Dependencies | Yes | We evaluate the scalability of Fed Scale Runtime, Fed ML (Git Hub commit@2ee0517) and Flower (v0.17.0 atop Ray v1.9.2) using a cluster with 10 GPU nodes. |
| Experiment Setup | Yes | We set the minibatch size of each participant to 32, and the number of local steps K to 20. We cherry-pick the hyper-parameters with grid search, ending up with an initial learning rate 0.04 for CV tasks and 5e-5 for NLP tasks. The learning rate decays by 0.98 every 10 training rounds. |