Acceleration via Symplectic Discretization of High-Resolution Differential Equations
Authors: Bin Shi, Simon S. Du, Weijie Su, Michael I. Jordan
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study first-order optimization algorithms obtained by discretizing ordinary differential equations (ODEs) corresponding to Nesterov s accelerated gradient methods (NAGs) and Polyak s heavy-ball method. We consider three discretization schemes: symplectic Euler (S), explicit Euler (E) and implicit Euler (I) schemes. We show that the optimization algorithm generated by applying the symplectic scheme to a high-resolution ODE proposed by Shi et al. [2018] achieves the accelerated rate for minimizing both strongly convex functions and convex functions. On the other hand, the resulting algorithm either fails to achieve acceleration or is impractical when the scheme is implicit, the ODE is low-resolution, or the scheme is explicit. |
| Researcher Affiliation | Academia | Bin Shi University of California, Berkeley binshi@berkeley.edu Simon S. Du Institute for Advanced Study ssdu@ias.edu Weijie J. Su University of Pennsylvania suw@wharton.upenn.edu Michael I. Jordan University of California, Berkeley jordan@cs.berkeley.edu |
| Pseudocode | No | The paper provides mathematical update equations for algorithms but does not include structured pseudocode blocks or sections explicitly labeled "Algorithm". |
| Open Source Code | No | The paper does not contain any statements about releasing code or links to a code repository. |
| Open Datasets | No | This paper is theoretical and does not involve experiments using datasets, thus no dataset availability information is provided. |
| Dataset Splits | No | This paper is theoretical and does not involve experiments with data, so no training, validation, or test splits are mentioned. |
| Hardware Specification | No | This is a theoretical paper and does not describe any experiments that would require hardware specifications. |
| Software Dependencies | No | This is a theoretical paper and does not describe any software dependencies with version numbers. |
| Experiment Setup | No | This is a theoretical paper and does not include details on experimental setup, hyperparameters, or training configurations. |