Federated Learning with Manifold Regularization and Normalized Update Reaggregation
Authors: Xuming An, Li Shen, Han Hu, Yong Luo
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments demonstrate that Fed MRUR can achieve a new state-of-the-art (SOTA) accuracy with less communication. In this section, we validate the effectiveness of the proposed Fed MRUR algorithm using the experimental results on CIFAR-10/100 [18] and Tiny Image Net [18]. |
| Researcher Affiliation | Collaboration | 1 School of Information and Electronics, Beijing Institute of Technology, China 2 JD Explore Academy, China 3 School of Computer Science, Wuhan University, China |
| Pseudocode | Yes | Algorithm 1 Fed MRUR |
| Open Source Code | No | The paper does not contain an explicit statement about releasing open-source code or a link to a repository for the described methodology. |
| Open Datasets | Yes | In this section, we validate the effectiveness of the proposed Fed MRUR algorithm using the experimental results on CIFAR-10/100 [18] and Tiny Image Net [18]. |
| Dataset Splits | Yes | The CIFAR-10 dataset consists of 50K training images and 10K testing images. All the images are with 32x32 resolution belonging to 10 categories. In the CIFAR-100 dataset, there are 100 categories of images with the same format as CIFAR-10. Tiny Image Net includes 200 categories of 100K training images and 10K testing images, whose resolutions are 64x64. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions software components like 'Res Net-18' and 'SAM optimizer' but does not provide specific version numbers for any software dependencies (e.g., PyTorch, TensorFlow, Python, CUDA versions). |
| Experiment Setup | Yes | For all algorithms on all datasets, following [1, 42], the local and global learning rates are set as 0.1 and 1.0, the learning rate decay is set as 0.998 per communication round and the weight decay is set as 5e-4. Res Net-18 together with group normalization is adopted as the backbone to train the model. The clients settings for different tasks are summarized in Table 1. Other optimizer hyperparameters are as follow: ρ = 0.5 for SAM, α = 0.1 for client momentum, γ = 0.005, σ = 10000.0 and β = 1 for manifold regularization. |