Random Shuffling Beats SGD after Finite Epochs
Authors: Jeff Haochen, Suvrit Sra
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We present the first non-asymptotic results for this problem, proving that after a reasonable number of epochs RANDOMSHUFFLE converges faster than SGD. Specifically, we prove that for strongly convex, second-order smooth functions, the iterates of RANDOMSHUFFLE converge to the optimal solution as O(1/T 2 + n3/T 3) |
| Researcher Affiliation | Academia | 1Institute for Interdisciplinary Information Sciences, Tsinghua University 2Massachusetts Institute of Technology. |
| Pseudocode | No | The paper describes the algorithms (SGD and RANDOMSHUFFLE) in text, but does not provide a formal pseudocode block or algorithm box. |
| Open Source Code | No | The paper does not include any statements or links indicating that open-source code for the described methodology is provided. |
| Open Datasets | No | The paper is theoretical and does not involve empirical studies or the use of datasets for training. |
| Dataset Splits | No | The paper is theoretical and does not involve empirical studies. Therefore, no dataset splits for validation are discussed. |
| Hardware Specification | No | The paper is theoretical and does not describe empirical experiments. Therefore, no hardware specifications used for running experiments are provided. |
| Software Dependencies | No | The paper is theoretical and focuses on mathematical proofs and analysis. It does not mention any specific software dependencies with version numbers required to reproduce experimental results. |
| Experiment Setup | No | The paper is theoretical and does not describe empirical experiments. Therefore, no experimental setup details like hyperparameters or training configurations are provided. |