Sharpness, Restart and Acceleration
Authors: Vincent Roulet, Alexandre d'Aspremont
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We illustrate our results by testing our adaptive restart methods, denoted Adap and Crit, introduced respectively in Sections 2.2 and 4 on several problems and compare them against simple gradient descent (Grad), accelerated gradient methods (Acc), and the restart heuristic enforcing monotonicity (Mono in [O Donoghue and Candes, 2015]). For Adap we plot the convergence of the best method found by grid search to compare with the restart heuristic. This implicitly assumes that the grid search is run in parallel with enough servers. For Crit we use the optimal f found by another solver. This gives an overview of its performance in order to potentially approximate it along the iterations in a future work as done with Polyak steps [Polyak, 1987]. |
| Researcher Affiliation | Academia | Vincent Roulet INRIA, ENS Paris France vincent.roulet@inria.fr, Alexandre d Aspremont CNRS, ENS Paris France aspremon@ens.fr |
| Pseudocode | Yes | Algorithm 1 Scheduled restarts for smooth convex minimization, Algorithm 2 Universal scheduled restarts for convex minimization, Algorithm 3 Restart on criterion |
| Open Source Code | No | The paper does not provide any explicit statements or links indicating that the source code for the methodology described is publicly available. |
| Open Datasets | Yes | In Figure 1, we solve classification problems with various losses on the UCI Sonar data set [Asuncion and Newman, 2007]. Asuncion, A. and Newman, D. [2007], Uci machine learning repository. |
| Dataset Splits | No | The paper mentions using the 'UCI Sonar data set' but does not provide specific details on how this dataset was split into training, validation, and test sets, nor does it specify any cross-validation setup. |
| Hardware Specification | No | The paper does not provide any specific details regarding the hardware used for running the experiments, such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | The paper does not provide specific software dependencies, such as library names with version numbers, required to replicate the experiments. |
| Experiment Setup | Yes | All restart schemes were done using the accelerated gradient with backtracking line search detailed in the Supplementary Material, with large dots representing restart iterations. Regularization parameters for dual SVM and LASSO were set to one. |