Single Loop Gaussian Homotopy Method for Non-convex Optimization
Authors: Hidenori Iwakiri, Yuhang Wang, Shinji Ito, Akiko Takeda
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Computational complexity analysis is performed on the SLGH algorithm under various situations: either a gradient or gradient-free oracle of a GH function can be obtained for both deterministic and stochastic settings. [...] In numerical experiments, our SLGH algorithms show faster convergence than an existing double loop GH method while outperforming gradient descent-based methods in terms of finding a better solution. |
| Researcher Affiliation | Collaboration | Hidenori Iwakiri The University of Tokyo, RIKEN AIP iwakiri-hidenori2020@g.ecc.u-tokyo.ac.jp Yuhang Wang The University of Tokyo utyuuhikou@gmail.com Shinji Ito NEC Corporation, RIKEN AIP i-shinji@nec.com Akiko Takeda The University of Tokyo, RIKEN AIP takeda@mist.i.u-tokyo.ac.jp |
| Pseudocode | Yes | Algorithm 1 Standard GH method ([20, 11]) Require: Objective function f, iteration number T, sequence {t1, . . . , t T } satisfying t1 > > t T . Find a solution x1 for minimizing F(x, t1). for k = 1 to T do Find a stationary point xk+1 of F(x, tk+1) with the initial solution xk. end for return x T |
| Open Source Code | No | The paper does not provide an explicit statement or link for open-source code release. |
| Open Datasets | Yes | The target models were well-trained DNNS for CIFAR-10 and MNIST, respectively. |
| Dataset Splits | No | The paper mentions using 'CIFAR-10' and 'MNIST' datasets but does not explicitly provide details on training, validation, or test splits (e.g., percentages, counts, or explicit reference to standard splits). |
| Hardware Specification | Yes | All experiments were conducted using Python and Tensorflow on Intel Xeon CPU and NVIDIA Tesla P100 GPU. |
| Software Dependencies | No | The paper mentions 'Python' and 'Tensorflow' but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | Please note that SLGHd takes approximately twice the computational time per iteration than the other algorithms because it needs additional queries for the computation of the derivative in terms of t. See Appendix E for a more detailed presentation of the experimental setup and results. |