Vanilla Bayesian Optimization Performs Great in High Dimensions
Authors: Carl Hvarfner, Erik Orm Hellsten, Luigi Nardi
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate that standard BO works drastically better than previously thought for high-dimensional tasks, outclassing existing high-dimensional BO algorithms on a wide range of real-world problems. Further, we aim to shed light on the inner workings of the BO machinery and why minimal changes in assumptions yield a dramatic increase in performance. The result is a performant vanilla BO algorithm for dimensionalities well into the thousands. 6.1. Sparse Synthetic Test Functions We start by evaluating the DSP on a collection of commonly considered synthetic test functions with varying total and effective dimensionality. 6.3. High-dimensional Optimization Tasks We now benchmark Vanilla BO with the DSP against a collection of frequently considered tasks in the high-dimensional literature |
| Researcher Affiliation | Collaboration | 1Lund University, Lund, Sweden 2DBTune, Malm o, Sweden. |
| Pseudocode | No | The paper describes algorithms and methods in textual form and uses mathematical equations, but it does not include any clearly labeled pseudocode blocks or algorithm listings. |
| Open Source Code | Yes | Our code is publicly available at https://github.com/hvarfner/ vanilla_bo_in_highdim. |
| Open Datasets | Yes | We start by evaluating the DSP on a collection of commonly considered synthetic test functions with varying total and effective dimensionality... The Lunar Lander (12D) and Robot Pushing (14D) tasks from (Wang et al., 2017), as well as the Swimmer (16D) and Hopper (32D) reinforcement learning tasks from the Mu Jo Co suite (Todorov et al., 2012)... Specifically, we consider MOPTA08 (124D), SVM (388D), Lasso-DNA (180D), and the Mu Jo Co (Todorov et al., 2012) Ant (888D) and Humanoid (6392D) reinforcement learning tasks. |
| Dataset Splits | No | The paper mentions initialization (e.g., 'initialized with 30 samples') and acquisition optimization budgets, but it does not explicitly state how the datasets were split into training, validation, or test sets. While it uses standard benchmarks, the specific split percentages or methodology for their experiments are not detailed within the paper. |
| Hardware Specification | No | The paper mentions 'a single model fitting takes upwards of 5 minutes on SVM on 4 CPUS for later iterations.' However, it does not specify the model or type of these CPUs, nor does it mention any GPU models or other detailed hardware specifications. |
| Software Dependencies | No | The paper mentions software tools like 'Log EI (Ament et al., 2023)', 'Bo Torch (Balandat et al., 2020)', and 'GPy Opt-authors, 2016', but it does not provide specific version numbers for these or any other software dependencies, such as Python or specific libraries. |
| Experiment Setup | Yes | We instantiate the DSP with µ0 ? 3, which equates to ℓ 0.50 for D 6 under the mode of ppℓq. We initialize all methods with 30 samples, marked by a dashed vertical line. On all benchmarks, we use Log EI (Ament et al., 2023), using a low acquisition optimization budget of 512 initial (global) SOBOL samples and 512 Gaussian samples around the incumbent, followed by L-BFGS on the 4 best candidates, which is made possible by the low-complexity-high-smoothness model. As such, we fix σ2 f 1 to match the scale of the standardized observations, and to ensure that σ2 f does not diminish over time. |