Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Gaussian Process Bandit Optimisation with Multi-fidelity Evaluations
Authors: Kirthevasan Kandasamy, Gautam Dasarathy, Junier B. Oliva, Jeff Schneider, Barnabas Poczos
NeurIPS 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we demonstrate that MF-GP-UCB outperforms single fidelity methods on a series of synthetic examples, three hyper-parameter tuning tasks and one inference problem in Astrophysics. |
| Researcher Affiliation | Academia | Carnegie Mellon University, Rice University EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1 MF-GP-UCB Inputs: kernel κ, bounds {ζ(m)}M m=1, thresholds {γ(m)}M m=1. |
| Open Source Code | Yes | Our matlab implementation and experiments are available at github.com/kirthevasank/mf-gp-ucb. |
| Open Datasets | Yes | We used the regression method from [14] on the 4-dimensional coal power plant dataset. We tuned the 6 hyper-parameters the regularisation penalty, the kernel scale and the kernel bandwidth for each dimension each in the range (10 3, 104) using 5-fold cross validation. This experiment used M = 3 and 2000, 4000, 8000 points at each fidelity. ... We use Type Ia supernovae data [7] for maximum likelihood inference on 3 cosmological parameters |
| Dataset Splits | Yes | We set this up as a M = 2 fidelity experiment with the entire training set at the second fidelity and 500 points at the first. Each query was 5-fold cross validation on these training sets. |
| Hardware Specification | No | The paper mentions 'CPU Time' in its results (e.g., Figure 4) but does not provide specific details on the CPU, GPU, or any other hardware used for the experiments. |
| Software Dependencies | No | The paper mentions 'Our matlab implementation' but does not specify the version of Matlab or any other software dependencies with version numbers. |
| Experiment Setup | Yes | Our implementation uses some standard techniques in Bayesian optimisation to learn the kernel such as initialisation with random queries and periodic marginal likelihood maximisation. ... To set γ(m) s we use the following intuition: if the algorithm, is stuck at fidelity m for too long then γ(m) is probably too small. We start with small values for γ(m). If the algorithm does not query above the mth fidelity for more than λ(m+1)/λ(m) iterations, we double γ(m). ... The goal is to tune the kernel bandwidth and the soft margin coefficient in the ranges (10 3, 101) and (10 1, 105) respectively on a dataset of size 2000. ... We tuned the 6 hyper-parameters the regularisation penalty, the kernel scale and the kernel bandwidth for each dimension each in the range (10 3, 104) using 5-fold cross validation. |