AdaGrad Avoids Saddle Points
Authors: Kimon Antonakopoulos, Panayotis Mertikopoulos, Georgios Piliouras, Xiao Wang
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | we resolve this challenge by combining a series of step-size stabilization arguments with a recursive representation of the ADAGRAD preconditioner that allows us to employ stable manifold techniques and ultimately show that the induced trajectories avoid saddle points from almost any initial condition. |
| Researcher Affiliation | Collaboration | 1Laboratory for Information and Inference Systems, IEM, STI, EPFL, 1015 Lausanne, Switzerland. 2This work was done when KA was with Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France. 3Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France 4Criteo AI Lab 5Singapore University of Technology and Design 6Shanghai University of Finance and Economics. |
| Pseudocode | No | The paper defines algorithms using mathematical equations (e.g., xt+1 = xt Γt f(xt)), but does not provide any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any concrete statement or link regarding the public availability of source code for the described methodology. |
| Open Datasets | No | This is a theoretical research paper focusing on mathematical analysis and proofs, and it does not involve empirical studies with datasets. Therefore, no information about publicly available training datasets is provided. |
| Dataset Splits | No | This is a theoretical research paper focused on mathematical analysis and proofs, not empirical experiments. Therefore, no dataset split information for training, validation, or testing is provided. |
| Hardware Specification | No | This is a theoretical research paper, and it does not describe any experimental setup involving specific hardware specifications. |
| Software Dependencies | No | This is a theoretical research paper focused on mathematical analysis and proofs. It does not describe any computational experiments or list software dependencies with version numbers. |
| Experiment Setup | No | This is a theoretical research paper that focuses on mathematical analysis and proofs of algorithm properties. It does not include an experimental setup with hyperparameters or training configurations. |