FedNest: Federated Bilevel, Minimax, and Compositional Optimization
Authors: Davoud Ataee Tarzanagh, Mingchen Li, Christos Thrampoulidis, Samet Oymak
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we numerically investigate the impact of several attributes of our algorithms on a hyper-representation problem (Franceschi et al., 2018), a hyper-parameter optimization problem for loss function tuning (Li et al., 2021), and a federated minimax optimization problem. |
| Researcher Affiliation | Academia | 1Email: tarzanaq@umich.edu, University of Michigan 2Emails: {mli176@,oymak@ece.}ucr.edu, University of California, Riverside 3Email: cthrampo@ece.ubc.ca, University of British Columbia. |
| Pseudocode | Yes | Algorithm 1 FEDNEST, Algorithm 2 x+ = FEDOUT, Algorithm 3 p N = FEDIHGP, Algorithm 4 y+ = FEDINN, Algorithm 5 x+ = LFEDOUT, Algorithm 6 y+ = LFEDINN |
| Open Source Code | Yes | FEDNEST code is available at https://github.com/ucr-optml/FedNest. |
| Open Datasets | Yes | Hyper-representation experiments on a 2-layer MLP and MNIST dataset. and Loss function tuning on a 3-layer MLP and imbalanced MNIST dataset to maximize class-balanced test accuracy. |
| Dataset Splits | Yes | and split each client s data evenly to train and validation datasets. Thus, each client has 300 train and 300 validation samples. and we employ 80%-20% train-validation on each client |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware (e.g., GPU models, CPU types, or memory specifications) used for running the experiments. |
| Software Dependencies | No | The paper states that its federated algorithm implementation is based on (Ji, 2018), implying the use of PyTorch, but it does not list specific version numbers for any software dependencies or libraries. |
| Experiment Setup | Yes | In Section 4, our federated algorithm implementation is based on (Ji, 2018), both hyper-representation and loss function tuning use batch size 64 and Neumann series parameter N = 5. We conduct 5 SGD/SVRG epoch of local updates in FEDINN and τ = 1 in FEDOUT. In FEDNEST, we use T = 1, have 100 clients in total, and 10 clients are selected in each FEDNEST epoch. |