Federated Learning via Meta-Variational Dropout
Authors: Insu Jeon, Minui Hong, Junhyeog Yun, Gunhee Kim
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conducted extensive experiments on various sparse and non-IID FL datasets. Meta VD demonstrated excellent classification accuracy and uncertainty calibration performance, especially for out-of-distribution (OOD) clients. |
| Researcher Affiliation | Academia | Insu Jeon Minui Hong Junhyeog Yun Gunhee Kim Seoul National University, Seoul, South Korea {insuj3on,antemrdm}@gmail.com {alsdml123,gunhee}@snu.ac.kr |
| Pseudocode | Yes | Algorithm 1 Meta VD algorithm with MAML and Reptile variant for FL |
| Open Source Code | Yes | Code is available at https://github.com/insujeon/Meta VD. ... Our code is available at https://github.com/insujeon/Meta VD. |
| Open Datasets | Yes | We used multiple FL datasets [72], including CIFAR-10, CIFAR-100, FEMINIST, and Celeb A. ... The CIFAR-10 and CIFAR-100 datasets [98] are popular for 10-class and 100-class image classification respectively. ... The Federated Extended MNIST (FEMNIST) is a widely used FL dataset for 62-class handwritten character recognition [72]. ... The Celeb A is a FL dataset based on [100] for 2-class image classification. |
| Dataset Splits | Yes | procedure LOCALADAPTATION_MAML(θ, α) ... Sample dataset Dm tr and Dm val from Dm |
| Hardware Specification | Yes | All experiments are run on a cluster of 32 NVIDIA GTX 1080 GPUs. |
| Software Dependencies | No | The paper mentions 'an optimization tool called Optuna[103]' but does not provide specific version numbers for general software dependencies like Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | The batch size was set to 64, and local steps was set to 5. Personalization was executed with a batch size of 64 and a 1-step update. ... The server learning rate η was explored within the range of [0.6, 0.7, 0.8, 0.9, 1.0]. The local SGD learning rate was investigated within the range of [0.005, 0.01, 0.015, 0.02, 0.025, 0.03]. ... For Meta VD, an additional KL divergence parameter β is needed, and we sought its optimal value within the range of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]. |