The Power of Extrapolation in Federated Learning
Authors: Hanmin Li, Kirill Acharya, Peter Richtarik
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our theory is corroborated with carefully constructed numerical experiments. Numerical evidence suggests that Fed Ex Prox achieves a 2 or higher speed-up in terms of iteration complexity compared to Fed Prox and improved performance compared to Fed Ex P. |
| Researcher Affiliation | Academia | Hanmin Li Gen AI Center of Excellence KAUST, Saudi Arabia hanmin.li@kaust.edu.sa Kirill Acharya Gen AI Center of Excellence KAUST, Saudi Arabia acharya.kk@phystech.edu Peter Richtárik Gen AI Center of Excellence KAUST, Saudi Arabia peter.richtarik@kaust.edu.sa Kirill was a student at MIPT during his internship at KAUST. |
| Pseudocode | Yes | Algorithm 1 Extrapolated SPPM (Fed Ex Prox) with partial client participation |
| Open Source Code | Yes | Our code is publicly available at the following link: https://anonymous.4open.science/r/Fed Ex Prox-F262/ |
| Open Datasets | No | The paper uses synthetic data for experiments, describing its generation: 'Each Ai is generated randomly from a uniform distribution between [0, 1), and the corresponding vector bi is also generated from the same uniform distribution.' It does not refer to a publicly available or open dataset with access information. |
| Dataset Splits | No | The paper describes an iterative optimization process for linear regression and does not specify training, validation, or test dataset splits in terms of percentages, sample counts, or predefined partitions for data. The problem is framed as minimizing a finite-sum objective function. |
| Hardware Specification | Yes | The code was run on a machine with AMD Ryzen 9 5900HX Radeon Graphics @ 3.3 GHz and 8 cores 16 threads. |
| Software Dependencies | No | The paper states: 'All the codes for the experiments are written in Python 3.11 with Num Py and Sci Py package.' While Python has a version, NumPy and SciPy packages are mentioned without specific version numbers, which are required for full reproducibility of dependencies. |
| Experiment Setup | Yes | The experiment settings specify various parameters: 'n = 30, d = 900' for the large dimension regime. For experimental comparisons, 'γ is picked from the set {0.0001, 0.001, 0.01, 0.1, 1, 10}' and 'the two algorithms are run for K = 10000 iterations'. The local step size for Fed Ex P is detailed as 'the largest possible value 1/6t Lmax'. |