Breaking the Convergence Barrier: Optimization via Fixed-Time Convergent Flows
Authors: Param Budhraja, Mayank Baranwal, Kunal Garg, Ashish Hota6115-6122
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate the accelerated convergence properties of the proposed schemes on a range of numerical examples against the state-of-the-art optimization algorithms. In this section, we present empirical results on optimizing (non-convex) functions that satisfy PL-inequality and training deep neural networks. |
| Researcher Affiliation | Collaboration | 1 Indian Institute of Technology Kharagpur 2 Tata Consultancy Services Research, Mumbai 3 University of California, Santa Cruz |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks, nor are there any clearly labeled algorithm sections or code-like formatted procedures. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described, lacking specific repository links, explicit code release statements, or mention of code in supplementary materials. |
| Open Datasets | Yes | MNIST (60000 training samples, 10000 test samples) (Le Cun et al. 1998), and CIFAR10 (50000 training samples, 10000 test samples) (Krizhevsky and Hinton 2009). |
| Dataset Splits | No | The paper specifies training and test sample counts for MNIST (60000 training, 10000 test) and CIFAR10 (50000 training, 10000 test), but does not provide specific details for a validation split or overall split percentages (e.g., train/validation/test percentages). |
| Hardware Specification | Yes | The algorithms were implemented using Py Torch 0.4.1 on a 16GB Core-i7 2.8GHz CPU and NVIDIA Ge Force GTX-1060 GPU. |
| Software Dependencies | Yes | The algorithms were implemented using Py Torch 0.4.1 |
| Experiment Setup | Yes | We use constant step-size for all the algorithms. The hyperparameters for different optimizers are tuned for optimal performance. ... Adam: β s = (0.9, 0.999), ϵ = 10 8 NAG: Momentum = 0.5 Fx TS-GF: β s = (1.25, 1.25), α s = (20, 1.98) Fx TS(M)-GF: β s = (1.25, 1.25), α s = (20, 1.98), Momentum = 0.18 and The learning rates for Adam and NAG are kept at 10 3. ... the learning rate for the Fx TS(M)-GF is chosen as 0.005. The momentum parameters for the NAG and Fx TS(M)-GF algorithms are chosen as 0.5 and 0.3, respectively. The loss function is the cross-entropy along with l2-regularization (coefficient 0.01). |