Sparsified SGD with Memory
Authors: Sebastian U. Stich, Jean-Baptiste Cordonnier, Martin Jaggi
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present numerical experiments to illustrate the theoretical findings and the good scalability for distributed applications. |
| Researcher Affiliation | Academia | Machine Learning and Optimization Laboratory (MLO) EPFL, Switzerland |
| Pseudocode | Yes | The pseudocode is given in Algorithm 1. |
| Open Source Code | Yes | Our code is open-source and publicly available at github.com/epfml/sparsified SGD. |
| Open Datasets | Yes | Datasets. We consider a dense dataset, epsilon [35], as well as a sparse dataset, RCV1 [20] where we train on the larger test set. |
| Dataset Splits | No | The paper mentions using datasets for training but does not provide specific details on training/validation/test splits, such as percentages, sample counts, or explicit splitting methodology. |
| Hardware Specification | Yes | Experiments were run on an Ubuntu 18.04 machine with a 24 cores processor Intel Xeon CPU E5-2680 v3 @ 2.50GHz. |
| Software Dependencies | No | The paper mentions 'Python3', 'numpy library', and 'scikit-learn' but does not provide specific version numbers for the libraries. |
| Experiment Setup | Yes | We study the convergence of the method using the stepsizes ηt = γ/(λ(t+a)) and hyperparameters γ and a set as in Table 2. |