Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
RMSprop converges with proper hyper-parameter
Authors: Naichen Shi, Dawei Li, Mingyi Hong, Ruoyu Sun
ICLR 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide empirical evidence for such a phase transition in our numerical experiments. |
| Researcher Affiliation | Academia | IOE, University of Michigan, EMAIL. Part of the work was done when Naichen Shi was working with Prof. Ruoyu Sun as an intern. ISE, University of Illinois at Urbana-Champaign. EMAIL ECE, University of Minnesota Twin Cities, EMAIL. University of Illinois at Urbana-Champaign. EMAIL. |
| Pseudocode | Yes | Algorithm 1 Randomly Shuffled Adam |
| Open Source Code | Yes | All codes generating experimental results are available on the Github repository https://github. com/soundsinteresting/RMSprop |
| Open Datasets | Yes | We conduct experiments of image classification and GAN on MNIST and CIFAR-10 to support our theoretical findings. |
| Dataset Splits | No | The paper mentions training and test sets but does not explicitly describe validation dataset splits. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used, such as CPU or GPU models. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers. |
| Experiment Setup | Yes | In the CIFAR experiments, we use Res Net-18. We choose β2 = 0.8, 0.9, 0.95, 0.99 respectively. With different batch sizes 8, 16, 32, we run each algorithm for 100 epochs without explicit regularization. |