Universal Average-Case Optimality of Polyak Momentum
Authors: Damien Scieur, Fabian Pedregosa
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | Our main contribution is to prove that any optimal average-case method converges in the number of iterations to PM, under mild assumptions. This brings a new perspective on this classical method, showing that PM is asymptotically both worst-case and average-case optimal. |
| Researcher Affiliation | Industry | 1Samsung SAIT AI Lab, Montreal 2Google Research. |
| Pseudocode | No | The paper defines the optimal average-case method using mathematical recurrence relations (equation 8) and Polyak momentum (PM) as equations, but it does not present these in a structured pseudocode or algorithm block. |
| Open Source Code | No | The paper does not contain any statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | No | The paper refers to mathematical distributions like "Marchenko-Pastur distribution" and "uniform distribution of eigenvalues" for theoretical analysis and simulations, not typical real-world public datasets that would be split for training. |
| Dataset Splits | No | The paper is theoretical with illustrative simulations, and therefore does not discuss training, validation, or test splits for datasets typically found in empirical machine learning research. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, cloud resources) used to conduct the simulations or any other part of the research. |
| Software Dependencies | No | The paper does not specify any software dependencies, libraries, or their version numbers that would be necessary to replicate the work, such as Python versions, deep learning frameworks, or numerical computation libraries. |
| Experiment Setup | No | The paper discusses parameters for mathematical distributions used in simulations (e.g., "different ratios r = d/n" and "different intervals"), but it does not provide specific experimental setup details such as hyperparameters, optimization settings, or training configurations typically found in papers describing empirical machine learning experiments. |