Efficient optimization of loops and limits with randomized telescoping sums
Authors: Alex Beatson, Ryan P Adams
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our adaptive RT estimators on a range of applications including meta-optimization of learning rates, variational inference of ODE parameters, and training an LSTM to model long sequences. Section 7 presents experimental results. |
| Researcher Affiliation | Academia | 1Department of Computer Science, Princeton University, Princeton, NJ, USA. |
| Pseudocode | Yes | Appendix A presents algorithm pseudocode. |
| Open Source Code | Yes | Code may be found at https://github.com/PrincetonLIPS/randomized_telescopes. |
| Open Datasets | Yes | We next experiment with meta-optimization of a learning rate on MNIST. Finally, we study a high-dimensional optimization problem: training an LSTM to model sequences on enwik8. |
| Dataset Splits | Yes | There are 205 unique tokens. We use the first 90M, 5M, and 5M characters as the training, evaluation, and test sets. |
| Hardware Specification | No | The paper does not explicitly describe any specific hardware components (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | Lotka-Volterra ODE: 'batch size of 64... and a learning rate of 0.01. Evaluation is performed with a batch size of 512.' MNIST: 'Optimization is performed with a batch size of 100.' 'The outer optimization is performed with a learning rate of 0.01.' enwik8 LSTM: 'The optimization is performed with a learning rate of 2.2.' General: 'the tuning frequency K is set to 5, and the exponential moving average weight α is set to 0.9.' |