Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Informed Initialization for Bayesian Optimization and Active Learning
Authors: Carl Hvarfner, David Eriksson, Eytan Bakshy, Maximilian Balandat
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments in active learning and Bayesian Optimization on synthetic and real-world BO tasks, demonstrating that HIPE outperforms standard initialization strategies in terms of predictive accuracy, hyperparameter identification, and subsequent optimization performance, particularly in large-batch, few-shot settings relevant to many real-world Bayesian Optimization applications. |
| Researcher Affiliation | Industry | Carl Hvarfner Meta EMAIL David Eriksson Meta EMAIL Eytan Bakshy Meta EMAIL Max Balandat Meta EMAIL |
| Pseudocode | No | The paper describes the methodology using mathematical equations and descriptive text, but it does not contain a dedicated pseudocode block or algorithm section with structured steps formatted like code. |
| Open Source Code | Yes | The code to run the experiments in the paper, including all the benchmarks and plotting to reproduce our results, is available at https://github.com/hipeneurips/HIPE. An easy-to-run, tested version of the algorithm and accompanying notebook is available open-source in Bo Torch. The HIPE acquisition is found at https://github.com/meta-pytorch/botorch/ blob/main/botorch_community/acquisition/bayesian_active_learning.py. |
| Open Datasets | Yes | We use three types of benchmarks: synthetic optimization test functions, surrogate-based hyperparameter optimization tasks from LCBench (Zimmer et al., 2021), and high-dimensional SVM hyperparameter optimization problems. LCBench tasks are GP surrogate models trained on 2,000 evaluations of multi-layer perceptrons (MLPs) on real-world datasets, using a Matern 3/2 kernel; all surrogates are included as part of our code release. The SVM benchmarks follow the setup in Ament et al. (2023a), based on the problem originally introduced in Eriksson and Jankowiak (2021). For these tasks, a Support Vector Regressor (Drucker et al., 1996) is fit to the CTSlice dataset, using a fixed subset of 5,000 data points, with 20% reserved for validation. |
| Dataset Splits | Yes | For these tasks [SVM benchmarks], a Support Vector Regressor (Drucker et al., 1996) is fit to the CTSlice dataset, using a fixed subset of 5,000 data points, with 20% reserved for validation. |
| Hardware Specification | Yes | All experiments were conducted using an NVIDIA A40 GPU cluster. |
| Software Dependencies | No | The paper lists software packages like GPy Torch, Bo Torch, Pyro, Py Torch, Num Py, Sci Py, Pandas, Matplotlib, Seaborn, and LCBench with their licenses but does not specify explicit version numbers for these software dependencies. |
| Experiment Setup | Yes | Unless otherwise specified, we draw 192 burn-in samples followed by 288 hyperparameter samples, retaining every 24th sample for evaluation. Our GP prior is adapted from Hvarfner et al. (2024) to better suit a fully Bayesian setting. Specifically, we set ยต0 0.75 and ฯ 0.75, resulting in โd LNp0.75 logp Dq{2, 0.75q. The noise standard deviation is modeled as ฯฮต LNp 5.5, 0.75q, and the constant mean parameter follows c Np0, 0.25q. Acquisition functions are optimized jointly over the batch using multi-start L-BFGS-B optimization with 4 random restarts and 384 initial samples drawn from a scrambled Sobol sequence. For our proposed HIPE method and relevant baselines (BALD and NIPV), Monte Carlo estimators use M 12 hyperparameter samples, T 1024 test points drawn uniformly from the search space, and N 128 predictive posterior samples. |