ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training
Authors: Hui-Po Wang, Sebastian Stich, Yang He, Mario Fritz
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive results on a broad range of architectures, including CNNs (VGG, Res Net, Conv Nets) and U-nets, and diverse tasks from simple classification to medical image segmentation show that our highly effective training approach saves up to 20% computation and up to 63% communication costs for converged models. |
| Researcher Affiliation | Academia | Hui-Po Wang 1 Sebastian U. Stich 1 Yang He 1 Mario Fritz 1 1CISPA Helmholz Center for Information Security, Germany. Correspondence to: Hui-Po Wang <hui.wang@cispa.de>. |
| Pseudocode | Yes | Algorithm 1 Prog Fed Progressive training in a Federated Learning setting |
| Open Source Code | Yes | Code is available at https: //github.com/a514514772/Prog Fed. |
| Open Datasets | Yes | Datasets, tasks, and models. We consider four datasets: CIFAR-10 (Krizhevsky et al., 2009), CIFAR100 (Krizhevsky et al., 2009), EMNIST (Cohen et al., 2017), and Bra TS (Menze et al., 2014; Bakas et al., 2017; 2018). |
| Dataset Splits | No | We consider four datasets: CIFAR-10 (Krizhevsky et al., 2009), CIFAR100 (Krizhevsky et al., 2009), EMNIST (Cohen et al., 2017), and Bra TS (Menze et al., 2014; Bakas et al., 2017; 2018)... For the federated settings, we train Conv Nets on CIFAR-10 and EMNIST (3400 clients, non-IID), Res Net-18 on CIFAR-100 (500 clients, non-IID), and 3DUnet (Sheller et al., 2020) on the Bra TS dataset (10 clients, IID). |
| Hardware Specification | No | The paper refers to 'limited hardware resources' on edge devices but does not provide specific details such as GPU models, CPU types, or memory used for running the experiments described in the study. |
| Software Dependencies | No | We implement all settings with Pytorch (Paszke et al., 2019). |
| Experiment Setup | Yes | In the centralized experiments, we implement models based on De Vries & Taylor (2017), where we run all experiments for 200 epochs and decay the learning rates in {60, 120, 160} epochs by a factor of 0.1. ... We run 1500 epochs for EMNIST, 2000 epochs for CIFAR-10, 3000 epochs for CIFAR100, and 100 epochs for Bra TS. We set S = 3 for EMNIST and S = 4 for all the other datasets and Ts as the practical guideline described in Section 3. We adopt 5 and 25 warm-up epochs for federated EMNIST and federated CIFAR-100, respectively. |