FedDAR: Federated Domain-Aware Representation Learning
Authors: Aoxiao Zhong, Hao He, Zhaolin Ren, Na Li, Quanzheng Li
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We have performed intensive empirical studies on both synthetic and real-world medical datasets which demonstrate its superiority over prior FL methods. Our code is available at https://github.com/zlz0414/Fed DAR. We validate our method s effectiveness on both synthetic and real datasets. We first experiment on the exact synthetic dataset described in our theoretical analysis to verify our theory. We then conduct experiments on a real dataset, Fair Face (Kärkkäinen & Joo, 2019), with controlled domain distributions to investigate the robustness of our algorithm under different levels of heterogeneity. Finally, we compare our method with various baselines on a real federated learning benchmark, EXAM (Dayan et al., 2021) with real-world domain distributions. We also conduct extensive ablation studies on it to discern the contribution of each component of our method. |
| Researcher Affiliation | Academia | Aoxiao Zhong1 3 , Hao He2 , Zhaolin Ren1, Na Li1, and Quanzheng Li3 1Harvard University 2Massachusetts Institute of Technology 3Massachusetts General Hospital/Harvard Medical School |
| Pseudocode | Yes | Algorithm 1 FEDDAR. Algorithm 2 FEDDAR for linear regression. |
| Open Source Code | Yes | Our code is available at https://github.com/zlz0414/Fed DAR. |
| Open Datasets | Yes | We use Fair Face (Kärkkäinen & Joo, 2019), a public face image dataset containing 7 race groups which are considered as the domains. We use the EXAM dataset (Dayan et al., 2021), a large-scale, real-world healthcare FL study. |
| Dataset Splits | Yes | We apply 5-fold cross-validation. All the models are trained for T = 20 communication rounds with Adam optimizer and a learning rate of 10 4. The models are evaluated by aggregating predictions on the local validation sets and then calculating the area under curve (AUC) for each domain. We also report the AUCs averaged on clients local validation set. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU models, or cloud instance types used for running experiments. It only mentions general computing environments. |
| Software Dependencies | No | The paper mentions using "Adam optimizer" and "Res Net-34" but does not specify software versions for these or other dependencies (e.g., Python, PyTorch, TensorFlow, CUDA versions). The codebase adaptation is mentioned but specific versions of libraries are not listed. |
| Experiment Setup | Yes | All the methods are trained for T = 100 communication rounds. We use Adam optimizer with a learning rate of 1 10 4 for the first 60 rounds and 1 10 5 for the last 40 rounds. The images are resized to 224 224 with only random horizontal flip for augmentation. The learning rate and the number of local epochs is tuned by grid search with a fixed batch size of 64. We tuned the projection dimension k for Fed DAR-SA among {4,8,16,32,64} with α = 1.0 and used k = 8 for all other α. All the models are trained for T = 20 communication rounds with Adam optimizer and a learning rate of 1 10 4. For each round we do 1 local epoch for all the methods. For all the methods, the models are initialized with the same pretrained model as in (Dayan et al., 2021) without any warmup. For Fed DAR-SA and Fed DARWA, we execute 5 epochs of update for heads on each round, and set representation dimension k = 16 for Fed DAR-SA. Hyperparameters including learning rate, number of epochs for head update and representation dimension are tuned through grid search with a fixed batch size of 36. |