Gradient Coding from Cyclic MDS Codes and Expander Graphs
Authors: Netanel Raviv, Rashish Tandon, Alex Dimakis, Itzhak Tamo
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally test our approach on Amazon EC2, and show that the generalization error of approximate gradient coding is very close to the full gradient while requiring significantly less computation from the workers. 7. Experimental Results In this section, we present results of experiments on our proposed approximate gradient coding schemes. |
| Researcher Affiliation | Collaboration | Netanel Raviv 1 Itzhak Tamo 2 Rashish Tandon 3 Alexandros G. Dimakis 4 1Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA. 2Department of Electrical Engineering Systems, Tel-Aviv University, Israel. 3Apple, Seattle, WA, USA. 4Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA. |
| Pseudocode | Yes | Algorithm 1 Gradient Coding 1: Input: Data S = {zi = (xi, yi)}m i=1, number of iterations T > 0, learning rate schedule ηt > 0, straggler tolerances (st)t [T ], a matrix B Cn n, and a function A : P(n) Cn. |
| Open Source Code | No | No concrete access to source code for the methodology was provided. The paper references an arXiv preprint (Raviv et al., 2017) which is not a code repository. |
| Open Datasets | Yes | The dataset we used was the Amazon Employee dataset from Kaggle. |
| Dataset Splits | No | While a validation set is mentioned ('AUC on a validation set'), specific split percentages, sizes, or methodology for creating the splits are not provided beyond '26,200 training samples'. |
| Hardware Specification | Yes | We ran our experiments using t2.micro worker instance types on Amazon EC2 and a c3.8xlarge master instance type. |
| Software Dependencies | No | The paper mentions 'implemented in python using MPI4py' but does not specify version numbers for these software dependencies, which is required for reproducibility. |
| Experiment Setup | Yes | For Gradient Coding we used a constant learning rate, chosen using crossvalidation. For Approximate Gradient Coding and Ignoring Stragglers, we used a learning rate of c1/(t + c2), which is typical for SGD, where c1 and c2 were also chosen via cross-validation. |