COLA: Decentralized Linear Learning
Authors: Lie He, An Bian, Martin Jaggi
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Here we illustrate the advantages of COLA in three respects: firstly we investigate the application in different network topologies and with varying subproblem quality ; secondly, we compare COLA with state-of-the-art decentralized baselines: 1 , DIGing [Nedic et al., 2017], which generalizes the gradient-tracking technique of the EXTRA algorithm [Shi et al., 2015], and 2 , Decentralized ADMM (aka. consensus ADMM), which extends the classical ADMM (Alternating Direction Method of Multipliers) method [Boyd et al., 2011] to the decentralized setting [Shi et al., 2014, Wei and Ozdaglar, 2013]; Finally, we show that COLA works in the challenging unreliable network environment where each node has a certain chance to drop out of the network. |
| Researcher Affiliation | Academia | EPFL lie.he@epfl.ch An Bian ETH Zurich ybian@inf.ethz.ch Martin Jaggi EPFL martin.jaggi@epfl.ch |
| Pseudocode | Yes | Algorithm 1: COLA: Communication-Efficient Decentralized Linear Learning |
| Open Source Code | Yes | Our implementation is publicly available under github.com/epfml/cola. |
| Open Datasets | Yes | Table 1: Datasets Used for Empirical Study URL 2M 3M 3.5e-5 Webspam 350K 16M 2.0e-4 Epsilon 400K 2K 1.0 RCV1 Binary 677K 47K 1.6e-3 |
| Dataset Splits | No | No specific details about train/validation/test splits are provided in the main text. The order of columns is mentioned to be shuffled before distribution, but no split ratios. |
| Hardware Specification | Yes | The decentralized network topology is simulated by running one thread per graph node, on a 2 12 core Intel Xeon CPU E5-2680 v3 server with 256 GB RAM. |
| Software Dependencies | No | We implement all algorithms in Py Torch with MPI backend. No specific version numbers for PyTorch or MPI are provided. |
| Experiment Setup | No | Due to space limit, details on the experimental configurations are included in Appendix D. These details are not present in the provided main text. |