Revisiting Weighted Aggregation in Federated Learning with Neural Networks
Authors: Zexi Li, Tao Lin, Xinyi Shang, Chao Wu
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments verify that our method can improve the generalization of the global model by a large margin on different datasets and models. |
| Researcher Affiliation | Academia | 1Zhejiang University, China. Zexi Li <zexi.li@zju.edu.cn>. 2Research Center for Industries of the Future, Westlake University, China. 3Xiamen University, China. Xinyi Shang <shangxinyi@stu.xmu.edu.cn>. Work was done during Xinyi s visit to Westlake University. Correspondence to: Chao Wu <chao.wu@zju.edu.cn>, Tao Lin <lintao@westlake.edu.cn>. |
| Pseudocode | Yes | Algorithm 1 FEDLAW: Federated Learning with Learnable Aggregation Weights |
| Open Source Code | No | Based on the above insights, we propose an effective method for Federated Learning with Learnable Aggregation Weights, named as FEDLAW ( source code). The phrase "( source code)" is present in the abstract, but it is not an active link or a clear URL to the code repository, indicating that concrete access is not provided in the paper itself. |
| Open Datasets | Yes | For concision, in section 4 and section 5, if not mentioned otherwise, we all use CIFAR-10 as the dataset and Simple CNN as the model. ... We conduct experiments to verify the effectiveness of FEDLAW. ... Dataset Fashion MNIST CIFAR-10 CIFAR-100. These are well-known public datasets commonly used in ML research. |
| Dataset Splits | No | The paper mentions using a "proxy dataset" for learning aggregation weights, and training on "trainset" and evaluating on "testset", but it does not specify a separate validation split for hyperparameter tuning or model selection during training. |
| Hardware Specification | Yes | We use 4 Quadro RTX 8000 GPUs for computation. |
| Software Dependencies | Yes | We conduct experiments under Python 3.8.5 and Pytorch 1.12.0. |
| Experiment Setup | Yes | We set the initial learning rates (LR) as 0.08 in CIFAR-10 and Fashion MNIST and set LR as 0.01 in CIFAR-100. We set a decaying LR scheduler in all experiments; that is, in each round, the local LR is 0.99*(LR of the last round). Local weight decay. We adopt local weight decay in all experiments. For CIFAR-10 and Fashion MNIST, we set the weight decay factor as 5e-4, and for CIFAR-100, we set it as 5e-5. Optimizer. We set SGD optimizer as the clients local solver and set momentum as 0.9. For the server-side optimizer (FEDDF, FEDBE, SERVER-FT, and FEDLAW), we use Adam optimizer and betas=(0.5, 0.999). |