Communication-Efficient Federated Bilevel Optimization with Global and Local Lower Level Problems
Authors: Junyi Li, Feihu Huang, Heng Huang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we validate the proposed algorithms through two real-world tasks: Federated Datacleaning and Federated Hyper-representation Learning. Empirical results show superior performance of our algorithms. |
| Researcher Affiliation | Academia | Junyi Li Computer Science University of Maryland College Park, MD 20742 junyili.ai@gmail.com Feihu Huang ECE University of Pittsburgh Pittsburgh, PA 15261 huangfeihu2018@gmail.com Heng Huang Computer Science University of Maryland College Park, MD 20742 henghuanghh@gmail.com |
| Pseudocode | Yes | Algorithm 1 Accelerated Federated Bilevel Optimization (Fed Bi OAcc) |
| Open Source Code | No | The paper mentions implementation details like Py Torch and Py Torch.Distributed but does not provide an explicit statement about releasing the source code for the described methodology or a link to a code repository. |
| Open Datasets | Yes | Dataset and Baselines. We create 10 clients and construct datasets based on MNIST [31]. For the training set, each client randomly samples 4500 images (no overlap among clients) from 10 classes and then randomly uniformly perturb the labels of ρ (0 ρ 1) percent samples. ... We consider the Omniglot [29] and Mini Image Net [44] data sets. |
| Dataset Splits | Yes | For the training set, each client randomly samples 4500 images (no overlap among clients) from 10 classes and then randomly uniformly perturb the labels of ρ (0 ρ 1) percent samples. For the validation set, each client randomly selects 50 clean images from a different class. |
| Hardware Specification | Yes | Our experiments were conducted on servers equipped with an AMD EPYC 7763 64-core CPU and 8 NVIDIA V100 GPUs. |
| Software Dependencies | No | The paper mentions 'Py Torch' and 'Py Torch.Distributed package' but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | Input: Constants cω, cν, cu, γ, η, τ, r; learning rate schedule {αt}, t [T], initial state (x1, y1, u1); ... The local step I is set as 5 for Fed Bi O, Fed Bi OAcc and Fed Avg. ... We perform grid search to find the best hyper-parameters for each method and report the best results. Specific choices are included in Appendix B.1. |