Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
When to Trust Aggregated Gradients: Addressing Negative Client Sampling in Federated Learning
Authors: Wenkai Yang, Yankai Lin, Guangxiang Zhao, Peng Li, Jie Zhou, Xu Sun
TMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on multiple image and text classification tasks validate the great effectiveness of our method in various settings. Our code is available at https://github.com/lancopku/FedGLAD. |
| Researcher Affiliation | Collaboration | Wenkai Yang EMAIL Center for Data Science Peking University Yankai Lin EMAIL Gaoling School of Artificial Intelligence Renmin University of China Guangxiang Zhao EMAIL Shanghai AI Lab Peng Li EMAIL Institute for AI Industry Research (AIR) Tsinghua University Jie Zhou EMAIL Pattern Recognition Center, We Chat AI Tencent Inc. Xu Sun EMAIL National Key Laboratory for Multimedia Information Processing, School of Computer Science Peking University |
| Pseudocode | Yes | Algorithm 1 Gradient Similarity Aware Learning Rate Adaptation for Federated Learning |
| Open Source Code | Yes | Our code is available at https://github.com/lancopku/FedGLAD. |
| Open Datasets | Yes | We perform main experiments on two image classification tasks: CIFAR-10 (Krizhevsky et al., 2009) and MNIST (Le Cun et al., 1998), and two text classification tasks: 20News Groups (Lang, 1995) and AGNews (Zhang et al., 2015). ... Furthermore, we conduct extra experiments on some realistic federated datasets to show that our method can also work well in these realistic scenarios, and the results are in Appendix S. ... Here, we conduct experiments on two realistic federated datasets: Federated EMNIST (Cohen et al., 2017; Reddi et al., 2020) (FEMNIST) and Stack Overflow. |
| Dataset Splits | Yes | To simulate realistic non-i.i.d. data partitions, we follow previous studies (Wang et al., 2019a; Lin et al., 2021) by taking the advantage of the Dirichlet distribution Dir(α). In our main experiments, we choose the non-i.i.d. degree hyper-parameter α as 0.1. ... Following previous studies (Reddi et al., 2020), we use all original training samples for federated training and use test examples for final evaluation. |
| Hardware Specification | Yes | All experiments are conducted on 4 TITAN RTX. |
| Software Dependencies | No | Our code is mainly based on Fed NLP (Lin et al., 2021) and Fed ML (He et al., 2020). ... We choose the client optimizers for image and text tasks as SGDM and Adam respectively. |
| Experiment Setup | Yes | The number of local epochs is 5 for two image classification datasets, and 1 for two text classification datasets. ... The optimal local learning rate ηl and batch size B which are optimal in the centralized training as the local training hyper-parameters for all federated baselines. ... The optimal training hyper-parameters (e.g., server learning rates) will be tuned based on the training loss of the previous 20% rounds (Reddi et al., 2020). ... The bounding factor γ is fixed as 0.02. The exponential decay factor β is fixed as 0.9. ... Table 16: Local training hyper-parameters for each dataset. ... Table 18: The optimal server learning rates for each baseline method in our experiments. |