Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
How Free is Parameter-Free Stochastic Optimization?
Authors: Amit Attia, Tomer Koren
ICML 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study the problem of parameter-free stochastic optimization, inquiring whether, and under what conditions, do fully parameter-free methods exist: these are methods that achieve convergence rates competitive with optimally tuned methods, without requiring significant knowledge of the true problem parameters. [...] In the non-convex setting, we demonstrate that a simple hyperparameter search technique results in a fully parameter-free method that outperforms more sophisticated state-of-the-art algorithms. We also provide a similar result in the convex setting with access to noisy function values under mild noise assumptions. Finally, assuming only access to stochastic gradients, we establish a lower bound that renders fully parameter-free stochastic convex optimization infeasible, and provide a method which is (partially) parameter-free up to the limit indicated by our lower bound. |
| Researcher Affiliation | Collaboration | 1Blavatnik School of Computer Science, Tel Aviv University 2Google Research Tel Aviv. |
| Pseudocode | Yes | Algorithm 1: Adaptive projected SGD tuning; Algorithm 2: Non-convex SGD tuning; Algorithm 3: Convex SGD tuning |
| Open Source Code | No | The paper does not contain any explicit statement about open-source code availability for the described methodology, nor does it provide a link to a code repository. |
| Open Datasets | No | This is a theoretical paper focused on algorithm design and analysis, and as such, it does not use or provide information about specific datasets for training. |
| Dataset Splits | No | This is a theoretical paper focused on algorithm design and analysis, and therefore does not discuss training, validation, or test splits of datasets. |
| Hardware Specification | No | This is a theoretical paper that focuses on mathematical analysis and algorithm design, and as such, it does not specify any hardware used for experiments. |
| Software Dependencies | No | This is a theoretical paper focused on mathematical analysis and algorithm design; therefore, it does not list specific software dependencies with version numbers. |
| Experiment Setup | No | This is a theoretical paper focused on algorithm design and analysis, and does not provide details of an experimental setup such as hyperparameters or training configurations. |