Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning
Authors: Jiaqi Wang, Chenxu Zhao, Lingjuan Lyu, Quanzeng You, Mengdi Huai, Fenglong Ma
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comprehensive experiments conducted on benchmark datasets demonstrate the efficacy and generalization ability of Fed Type across diverse settings. We perform comprehensive experiments on multiple benchmark datasets, evaluating the Fed Type framework in both heterogeneous and homogeneous scenarios against state-of-the-art baselines. The experimental results consistently affirm the effectiveness of the Fed Type framework, underscoring its practicality and robust performance in real-world applications. |
| Researcher Affiliation | Collaboration | 1Pennsylvania State University 2Iowa State University 3Sony AI 4Byte Dance. |
| Pseudocode | Yes | Algorithm 1: Epoch-Level Algorithm Flow of UARL. Algorithm 2: Algorithm Flow of Fed Type. Algorithm 3: Algorithm Flow of Cmodel Training with Eq. (5). |
| Open Source Code | Yes | 1The code is available at https://github.com/ Jackqq Wang/Fed Type. |
| Open Datasets | Yes | We assess the effectiveness of the proposed Fed Type approach through image classification tasks conducted in the cross-device scenario on FMNIST, CIFAR-10, and CIFAR-100 datasets, and cross-silo scenario on Fed-ISIC19 dataset (Ogier du Terrail et al., 2022). |
| Dataset Splits | Yes | Our data distribution process involves initially allocating data to clients, followed by a subsequent split into local model training, testing, and conformal learning sets in a 7:2:1 ratio. |
| Hardware Specification | Yes | All experiments are conducted on an NVIDIA A100 with CUDA version 12.0, running on a Ubuntu 20.04.6 LTS server. |
| Software Dependencies | Yes | All baselines and the proposed Fed Type are implemented using Py Torch 2.0.1. |
| Experiment Setup | Yes | Our experimental setup involves 100 communication rounds, 100 clients, a 20% sample ratio for the cross-device experiments, and five local training epochs. ... For the local update, we set the learning rate as 0.0001, the batch size is 16, and the optimizer used in the optimization is Adam. Following the provided value in the work (Angelopoulos et al., 2020), we set λ = 0.5, κreg = 5, and θ = 0.1 for the local conformal model and proxy conformal model in Eq. (6), the conformal learning batch size is 32. The learning rate in the Platt scaling process is 0.01, and the maximum iteration is 10 following the default setting. |