DADAO: Decoupled Accelerated Decentralized Asynchronous Optimization
Authors: Adel Nabli, Edouard Oyallon
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Thus, we simultaneously obtain an accelerated rate for both computations and communications, leading to an improvement over stateof-the-art works, our simulations further validating the strength of our relatively unconstrained method. All our experiments are reproducible, using Py Torch (Paszke et al., 2019), our code being online 1. |
| Researcher Affiliation | Academia | 1Sorbonne Université, CNRS, ISIR, Paris, France 2Mila, Concordia University, Montréal, Canada. |
| Pseudocode | Yes | Algorithm 1: This algorithm block describes our implementation on each local machine. The ODE routine is described by Eq. 8. Algorithm 2: Pseudo-code of our implementation of DADAO on a single machine. |
| Open Source Code | Yes | All our experiments are reproducible, using Py Torch (Paszke et al., 2019), our code being online 1. https://github.com/Adel Nabli/DADAO/ |
| Open Datasets | No | We perform the empirical risk minimization for the decentralized linear regression task given by: Pj=1 aijx cij 2, (9) where aij Rd, and cij R correspond to m local data points stored at node i. We follow a protocol similar to (Kovalev et al., 2021a): we generate n independent synthetic datasets with the make regression functions of scikit-learn (Pedregosa et al., 2011), each worker storing m = 100 data points. |
| Dataset Splits | No | The paper states that synthetic datasets were generated and data points per worker, but no explicit train/validation/test splits with percentages or counts are provided. It mentions using 'standard experimental setting' but no specific split information. |
| Hardware Specification | No | This work was granted access to the HPC/AI resources of IDRIS under the allocation AD011013743 made by GENCI. (This mentions a resource type but no specific hardware models, CPUs, GPUs, or memory details.) |
| Software Dependencies | No | All our experiments are reproducible, using Py Torch (Paszke et al., 2019), our code being online 1. (This cites PyTorch with its original publication year, but does not provide a specific version number like 'PyTorch 1.9' for the software used in the experiments.) |
| Experiment Setup | No | We systematically used the proposed hyper-parameters of each reference paper for our implementation without any specific fine-tuning. (This indicates hyperparameters were used, but they are not explicitly listed or described for the current experiments within the paper's text.) |