FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction
Authors: Samiul Alam, Luyang Liu, Ming Yan, Mi Zhang
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the performance of Fed Rolex under two regimes: i) small-model small-dataset regime, and ii) large-model large-dataset regime |
| Researcher Affiliation | Collaboration | 1Michigan State University, 2The Ohio State University, 3Google Research, 4The Chinese University of Hong Kong, Shenzhen |
| Pseudocode | Yes | The pseudocode of Fed Rolex is in Algorithm 1. |
| Open Source Code | Yes | Our code is available at: https://github.com/AIo TMLSys-Lab/Fed Rolex. |
| Open Datasets | Yes | We train pre-activated Res Net18 (Pre Res Net18) models [26] on CIFAR-10 and CIFAR-100 [27]. ... For Stack Overflow, we followed [2] to use a 10% dropout rate to prevent overfitting, and 200 clients are randomly selected from a pool of 342, 477 clients in each communication round. The statistics of the datasets are listed in Table 2. |
| Dataset Splits | Yes | Table 2: Dataset statistics. ... Stack Overflow: Validation Clients 38,758, Validation Examples 16,491,230 |
| Hardware Specification | Yes | We implemented Fed Rolex and PT-based baselines using Py Torch [31] and Ray [32], and conducted our experiments on 8 NVIDIA A6000 GPUs. |
| Software Dependencies | No | The paper mentions using 'Py Torch [31] and Ray [32]' but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | The details of the hyper-parameters for model training are included in Appendix A.7. ... To ensure a fair comparison, all the PT-based baselines are trained using the same learning rate, number of communication rounds, and multi-step learning rate decay schedule. The details of the schedule for each dataset and experiment are described in Appendix A.7. |