UnderGrad: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees
Authors: Kimon Antonakopoulos, Dong Quan Vu, Volkan Cevher, Kfir Levy, Panayotis Mertikopoulos
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5. Numerical Experiments For the experimental validation of our results, we focus on the simplex setup of Example 1 with linear losses and d = 100. Our first experiment concerns the perfect SFO case and tracks down the convergence properties of UNDERGRAD run with the entropic regularizer adapted to the simplex. As a baseline, we ran UNIXGRAD, also with the entropic regularizer. |
| Researcher Affiliation | Collaboration | 1Laboratory for Information and Inference Systems, IEL STI EPFL, 1015 Lausanne, Switzerland. 2This work was done when KA and DQV were with Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France. 3Safran Tech, Magny-Les-Hameaux, France. 4Technion, Haifa, Israel. 5A Viterbi Fellow. 6Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France. 7Criteo AI Lab. |
| Pseudocode | Yes | Algorithm 1: Universal dual extrapolation with reweighted gradients (UNDERGRAD) |
| Open Source Code | Yes | The code is available at https://github.com/dongquan-vu/UnDerGrad_Universal_CnvOpt_ICML2022. |
| Open Datasets | No | For the experimental validation of our results, we focus on the simplex setup of Example 1 with linear losses and d = 100. The paper describes a problem setup for experimentation but does not specify a publicly available dataset with access information. |
| Dataset Splits | No | For the experimental validation of our results, we focus on the simplex setup of Example 1 with linear losses and d = 100. The paper does not specify training, validation, or test dataset splits, as its experiments focus on algorithm convergence rather than model generalization on fixed data splits. |
| Hardware Specification | No | The paper does not provide any specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not list any specific software libraries, frameworks, or programming languages with version numbers used for the experiments. |
| Experiment Setup | Yes | For the experimental validation of our results, we focus on the simplex setup of Example 1 with linear losses and d = 100. Our first experiment concerns the perfect SFO case and tracks down the convergence properties of UNDERGRAD run with the entropic regularizer adapted to the simplex. As a baseline, we ran UNIXGRAD, also with the entropic regularizer. A first challenge here is that the Bregman diameter Bh of the simmplex is infinite, so UNIXGRAD is not well-defined. On that account, we choose the step-size update rule of UNIXGRAD such that its initial step-size γ1 equals the initial learning rate η1 of UNDERGRAD. We also ran UNIXGRAD with the update rule such that γ1 is smaller or larger than η1. |