Communication-Efficient Federated Group Distributionally Robust Optimization
Authors: Zhishuai Guo, Tianbao Yang
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The effectiveness of our algorithms has been demonstrated on a variety of real-world tasks, including natural language processing and computer vision. |
| Researcher Affiliation | Academia | Zhishuai Guo, Tianbao Yang Department of Computer Science and Engineering Texas A&M University zhishguo@tamu.edu,tianbao-yang@tamu.edu |
| Pseudocode | Yes | Algorithm 1 FGDRO-CVa R |
| Open Source Code | No | All used data are publicly available. Code will be released later. |
| Open Datasets | Yes | We use Pile [15], Civil Comments [5], Camelyon17 [1], i Wild Cam2020 [3], and Poverty [74]. |
| Dataset Splits | Yes | Camelyon17 focuses on tumor detection from lymph node images [1], with data from five hospitals split into training (3), validation (1), and testing (1) sets. |
| Hardware Specification | Yes | Each algorithm was run on a high performance cluster where each machine uses a NVIDIA A100 GPU. |
| Software Dependencies | No | The paper mentions specific models and frameworks (e.g., GPT2, Distil BERT, Huggingface, DenseNet, ResNet) but does not provide specific version numbers for programming languages, libraries, or other software dependencies. |
| Experiment Setup | Yes | We tune the initial step size in [1e-4, 1e-3, 1e-2, 1e-1]. All algorithms set the communication interval I = 32 unless otherwise specified. The local mini-batch sizes are set to 32. Experiments are run for 20K local iterations except for Pile, which runs for 200K iterations. The β parameters of FGDRO-KL and FGDRO-KL-Adam are tuned in [0.01, 0.1, 0.2, 0.5]. |