ADOM: Accelerated Decentralized Optimization Method for Time-Varying Networks
Authors: Dmitry Kovalev, Egor Shulgin, Peter Richtarik, Alexander V Rogozin, Alexander Gasnikov
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section we perform experiments with logistic regression for binary classification with ℓ2 regularization. That is, our loss function has the form j=1 log(1 + exp( bija ijx)) + r 2 x 2, (19) where aij Rd and bij { 1, +1} are data points and labels, r > 0 is a regularization parameter, and m is the number of data points stored on each node. In our experiments we use function sklearn.datasets.make classification from scikit-learn library for dataset generation. We generate a number of datasets consisting of 10,000 samples, distributed to the n = 100 nodes of the network with m = 100 samples on each node. We vary r to obtain different values of the condition number κ. We also vary the number of features d. |
| Researcher Affiliation | Academia | 1King Abdullah University of Science and Technology, Thuwal, Saudi Arabia 2Moscow Institute of Physics and Technology, Dolgoprudny, Russia 3Higher School of Economics, Moscow, Russia. |
| Pseudocode | Yes | Algorithm 1 PNGD: Projected Nesterov Gradient Descent; Algorithm 2 ADOM: Accelerated Decentralized Optimization Method |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the methodology described is publicly available. |
| Open Datasets | No | In our experiments we use function sklearn.datasets.make classification from scikit-learn library for dataset generation. We generate a number of datasets consisting of 10,000 samples, distributed to the n = 100 nodes of the network with m = 100 samples on each node. |
| Dataset Splits | No | The paper mentions generating datasets and distributing samples but does not specify any train/validation/test splits or percentages. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments. |
| Software Dependencies | No | The paper mentions using "sklearn.datasets.make classification from scikit-learn library" and cites "LIBSVM" but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | We vary r to obtain different values of the condition number κ. We also vary the number of features d. ... To calculate the dual gradient F (z) we use T = 3 steps of AGD in ADOM and T = 30 steps of AGD in DNM. ... We switch between 2 networks every t iterations, where t {50, 20, 10, 5}. ... We use T = 1 iterations of GD to calculate F (zk g) in ADOM. |