Linear Convergent Decentralized Optimization with Compression
Authors: Xiaorui Liu, Yao Li, Rongrong Wang, Jiliang Tang, Ming Yan
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 NUMERICAL EXPERIMENT We consider three machine learning problems ℓ2-regularized linear regression, logistic regression, and deep neural network. The proposed LEAD is compared with QDGD (Reisizadeh et al., 2019a), Deep Squeeze (Tang et al., 2019a), CHOCO-SGD (Koloskova et al., 2019), and two non-compressed algorithms DGD (Yuan et al., 2016) and NIDS (Li et al., 2019). |
| Researcher Affiliation | Academia | 1 Department of Computer Science and Engineering 2 Department of Mathematics 3 Department of Computational Mathematics, Science and Engineering Michigan State University, East Lansing, MI 48823, USA |
| Pseudocode | Yes | Algorithm 1 LEAD Input: Stepsize η, parameter (α, γ), X0, H1, D1 = (I W)Z for any Z Output: XK or 1/n Pn i=1 XK i |
| Open Source Code | No | The paper does not provide any explicit statement or link regarding the release of open-source code for the described methodology. |
| Open Datasets | Yes | Logistic regression. We further consider a logistic regression problem on the MNIST dataset. Neural network. We empirically study the performance of LEAD in optimizing deep neural network by training Alex Net (240 MB) on CIFAR10 dataset. |
| Dataset Splits | No | The paper mentions using MNIST and CIFAR10 datasets for training and testing but does not explicitly provide the specific percentages or sample counts for train/validation/test splits, nor does it refer to standard predefined splits with citations for reproducibility. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies (e.g., library names with version numbers like Python 3.8, PyTorch 1.9, or specific solver versions) needed to replicate the experiment. |
| Experiment Setup | Yes | For all experiments, we tune the stepsize η from {0.01, 0.05, 0.1, 0.5}. For QDGD, CHOCO-SGD and Deepsqueeze, γ is tuned from {0.01, 0.1, 0.2, 0.4, 0.6, 0.8, 1.0}. ... For LEAD, we simply fix α = 0.5 and γ = 1.0 for all experiments... The mini-batch size is 64 for each agents. |