Generalized Federated Learning via Sharpness Aware Minimization
Authors: Zhe Qu, Xingyu Li, Rui Duan, Yao Liu, Bo Tang, Zhuo Lu
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Theoretically, we show the convergence analysis of these two algorithms and demonstrate the generalization bound of Fed SAM. Empirically, our proposed algorithms substantially outperform existing FL studies and significantly decrease the learning deviation. |
| Researcher Affiliation | Academia | 1Department of Electrical Engineering, University of South Florida, Tampa, USA 2Department of Electrical and Computer Engineering, Mississippi State University, Starkville, USA 3Department of Computer Science and Engineering, University of South Florida, Tampa, USA. |
| Pseudocode | Yes | The pesudocode of Fed Avg is shown in Algorithm 1. |
| Open Source Code | No | The paper does not provide a specific repository link or an explicit statement about the release of its own source code. |
| Open Datasets | Yes | We use three images datasets: EMNIST (Cohen et al., 2017), CIFAR-10, and CIFAR-100 (Krizhevsky et al., 2009). |
| Dataset Splits | Yes | We train global models using a set of participating clients and examine their performance both on training and validation datasets. |
| Hardware Specification | Yes | We ran the experiments on a CPU/GPU cluster, with RTX 2080Ti GPU |
| Software Dependencies | No | The paper mentions using 'Py Torch (Paszke et al., 2019)' but does not provide a specific version number for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | Our cross-device FL setting includes 100 clients in total with participation rate 20%... the number of local epochs is set as K = 10 by default... The learning rates are individually tuned and other optimizer hyper-parameters such as ρ = 0.5 for SAM and β = 0.1 for momentum, unless explicitly stated otherwise. |