Lightweight Projective Derivative Codes for Compressed Asynchronous Gradient Descent
Authors: Pedro J Soto, Ilia Ilmer, Haibin Guan, Jun Li
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experiments were run on AWS Spot Instances; the workers are AWS EC2 c5a.large instances (compute optimized) and master is an AWS EC2 r3.large instance (Memory Optimized). The experimental procedure was written using Mpi4py (Dalc ın et al., 2005; 2008) in Python. We used a modification of the code in (Tandon, 2017) written by the first author of (Tandon et al., 2017) to implement Gradient Coding (GC) as well as the random data generation; the implementation only supported logistic regression and we generalized it to support multinomial logistic regression (i.e., more than one class). |
| Researcher Affiliation | Academia | 1Department of Computer Science, The Graduate Center, CUNY, New York, USA 2Icahn School of Medicine at Mount Sinai, New York, USA 3Department of Computer Science, CUNY Queens College & Graduate Center, New York, USA. |
| Pseudocode | Yes | Algorithm 1 Data Partition Assignment |
| Open Source Code | No | The paper references 'Tandon, R. gradient coding. https://github.com/rashisht1/gradient_coding, 2017.' and states 'We used a modification of the code in (Tandon, 2017) written by the first author of (Tandon et al., 2017) to implement Gradient Coding (GC)'. This refers to code *used* from another source, not open-source code for the LWPD method developed in this paper. |
| Open Datasets | No | The software in (Tandon, 2017) used a Gaussian mixture model of two distributions to create input features for the logistic model; we generalized it to allow for an arbitrary number of Gaussian distributions in the mixture to create a robust data set. The paper does not provide access information for this dataset. |
| Dataset Splits | No | The paper mentions 'testing/validation set error or loss' and that 'the validation error is the more important measure', but it does not provide specific numerical split percentages or sample counts for the training, validation, and test datasets. |
| Hardware Specification | Yes | The experiments were run on AWS Spot Instances; the workers are AWS EC2 c5a.large instances (compute optimized) and master is an AWS EC2 r3.large instance (Memory Optimized). |
| Software Dependencies | No | The paper mentions 'The experimental procedure was written using Mpi4py (Dalc ın et al., 2005; 2008) in Python.' but does not specify version numbers for Mpi4py, Python, or other key libraries/dependencies used. |
| Experiment Setup | No | The paper mentions architectural details like 'Re LU activations at the hidden nodes, Softmax at the output nodes, and cross entropy as the loss function' for deep neural networks experiments, but it does not provide specific hyperparameter values such as learning rate, batch size, or number of epochs, nor detailed optimizer settings. |