Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives
Authors: Michael Muehlebach, Michael I. Jordan
JMLR 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We analyze the convergence rate of various momentum-based optimization algorithms from a dynamical systems point of view. Our analysis exploits fundamental topological properties, such as the continuous dependence of iterates on their initial conditions, to provide a simple characterization of convergence rates. In many cases, closed-form expressions are obtained that relate algorithm parameters to the convergence rate. The analysis encompasses discrete time and continuous time, as well as time-invariant and time-variant formulations, and is not limited to a convex or Euclidean setting. In addition, the article rigorously establishes why symplectic discretization schemes are important for momentum-based optimization algorithms, and provides a characterization of algorithms that exhibit accelerated convergence. |
| Researcher Affiliation | Academia | Michael Muehlebach EMAIL Department of Electrical Engineering and Computer Sciences Department of Statistics University of California Berkeley, CA 94720-1776, USA Michael I. Jordan EMAIL Department of Electrical Engineering and Computer Sciences Department of Statistics University of California Berkeley, CA 94720-1776, USA |
| Pseudocode | No | The paper describes mathematical dynamics and discretizations like (13) in prose and equations, but does not present any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements or links indicating that source code for the described methodology is publicly available. |
| Open Datasets | No | The paper is theoretical, analyzing the convergence rate of optimization algorithms, and does not involve experiments or the use of specific datasets. |
| Dataset Splits | No | The paper is theoretical and does not present experimental results based on datasets, therefore, no dataset split information is provided. |
| Hardware Specification | No | The paper is theoretical and focuses on mathematical analysis of optimization algorithms; it does not describe any experimental setup that would require hardware specifications. |
| Software Dependencies | No | The paper is theoretical and does not detail an implementation or experimental setup; consequently, no specific software dependencies or their version numbers are provided. |
| Experiment Setup | No | The paper presents theoretical analysis of optimization algorithms and does not include an experimental section with specific setup details such as hyperparameters or training configurations. |