FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction
Authors: Feijie Wu, Xingchen Wang, Yaqing Wang, Tianci Liu, Lu Su, Jing Gao
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments are conducted on various datasets to showcase the superior performance of the proposed FIARSE. |
| Researcher Affiliation | Collaboration | 1Purdue University 2Google Deep Mind |
| Pseudocode | Yes | Algorithm 1 FIARSE Input: local learning rate ηl, global learning rate ηs, local updates K, initial model x0. |
| Open Source Code | Yes | Our code is released at https://github.com/Harli Wu/FIARSE. |
| Open Datasets | Yes | We evaluate the proposed methodology using a combination of two computer vision (CV) datasets and one natural language processing (NLP) dataset. Specifically, we employ CIFAR-10 and CIFAR-100 datasets [37] for image classification, and the AGNews dataset [79] for text classification. |
| Dataset Splits | Yes | For CIFAR-10 and CIFAR-100, we follow [22, 30] and partition the datasets into 100 clients based on a Dirichlet distribution setting α = 0.3. As for AGNews, we partition the datasets for 200 clients with Dirichlet distribution as well, but it is with the parameter of α = 1.0. |
| Hardware Specification | Yes | We train the neural network and run the program on a server with 8 NVIDIA A6000 GPUs, an Intel Xeon Gold 6254 CPU, and 256GB RAM. |
| Software Dependencies | Yes | Our codes are running with Python 3.7 and Pytorch 1.8.1. |
| Experiment Setup | Yes | Implementation. In this setting, we set the participation ratio to 10% by default. We perform 800-round training on the CV tasks while running for 300 rounds on the NLP task. To avoid the randomness of the results, we averaged the results from three different random seeds... Table 3 lists the hyperparameters that we use in the experiments. |