Gaussian Process Bandit Optimisation with Multi-fidelity Evaluations
Authors: Kirthevasan Kandasamy, Gautam Dasarathy, Junier B. Oliva, Jeff Schneider, Barnabas Poczos
NeurIPS 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we demonstrate that MF-GP-UCB outperforms single fidelity methods on a series of synthetic examples, three hyper-parameter tuning tasks and one inference problem in Astrophysics. |
| Researcher Affiliation | Academia | Carnegie Mellon University, Rice University {kandasamy, joliva, schneide, bapoczos}@cs.cmu.edu, gautamd@rice.edu |
| Pseudocode | Yes | Algorithm 1 MF-GP-UCB Inputs: kernel κ, bounds {ζ(m)}M m=1, thresholds {γ(m)}M m=1. |
| Open Source Code | Yes | Our matlab implementation and experiments are available at github.com/kirthevasank/mf-gp-ucb. |
| Open Datasets | Yes | We used the regression method from [14] on the 4-dimensional coal power plant dataset. We tuned the 6 hyper-parameters the regularisation penalty, the kernel scale and the kernel bandwidth for each dimension each in the range (10 3, 104) using 5-fold cross validation. This experiment used M = 3 and 2000, 4000, 8000 points at each fidelity. ... We use Type Ia supernovae data [7] for maximum likelihood inference on 3 cosmological parameters |
| Dataset Splits | Yes | We set this up as a M = 2 fidelity experiment with the entire training set at the second fidelity and 500 points at the first. Each query was 5-fold cross validation on these training sets. |
| Hardware Specification | No | The paper mentions 'CPU Time' in its results (e.g., Figure 4) but does not provide specific details on the CPU, GPU, or any other hardware used for the experiments. |
| Software Dependencies | No | The paper mentions 'Our matlab implementation' but does not specify the version of Matlab or any other software dependencies with version numbers. |
| Experiment Setup | Yes | Our implementation uses some standard techniques in Bayesian optimisation to learn the kernel such as initialisation with random queries and periodic marginal likelihood maximisation. ... To set γ(m) s we use the following intuition: if the algorithm, is stuck at fidelity m for too long then γ(m) is probably too small. We start with small values for γ(m). If the algorithm does not query above the mth fidelity for more than λ(m+1)/λ(m) iterations, we double γ(m). ... The goal is to tune the kernel bandwidth and the soft margin coefficient in the ranges (10 3, 101) and (10 1, 105) respectively on a dataset of size 2000. ... We tuned the 6 hyper-parameters the regularisation penalty, the kernel scale and the kernel bandwidth for each dimension each in the range (10 3, 104) using 5-fold cross validation. |