Bayesian Optimisation over Multiple Continuous and Categorical Inputs
Authors: Binxin Ru, Ahsan Alvi, Vu Nguyen, Michael A. Osborne, Stephen Roberts
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate empirically that our method outperforms existing approaches on both synthetic and realworld optimisation tasks with continuous and categorical inputs. We tested all these methods on a diverse set of synthetic and real problems: |
| Researcher Affiliation | Academia | 1University of Oxford. |
| Pseudocode | Yes | Algorithm 1 Co Ca BO Algorithm and Algorithm 2 Co Ca BO batch selection are provided in the paper. |
| Open Source Code | Yes | For all the baseline methods, we used their publicly available Python packages3. ... 4https://github.com/rubinxin/Co Ca BO_code |
| Open Datasets | Yes | SVM-Boston outputs the negative mean square test error of using a support vector machine (SVM) for regression on the Boston housing dataset (Dua & Graff, 2017); XG-MNIST returns classification test accuracy of a XGBoost model (Chen & Guestrin, 2016) on MNIST (Le Cun & Cortes, 2010); NAS-CIFAR10 performs the architecture search on convolutional neural network topology for CIFAR10 classification. We conducted the search using the NAS-Bench-101 dataset (Ying et al., 2019). |
| Dataset Splits | No | The paper describes the use of initial random points and how models are trained for predictive performance (e.g., "We train each model on 250 uniformly randomly sampled data points and evaluate the predictive log likelihood on 100 test data points"). However, it does not provide specific details on explicit training, validation, or test dataset splits in the context of the main Bayesian Optimization experiments, or how internal validation (if any) is handled. |
| Hardware Specification | Yes | For all the problems, the continuous inputs were normalised to x [ 1, 1]d and all experiments were conducted on a 36-core 2.3GHz Intel Xeon processor with 512 GB RAM. |
| Software Dependencies | No | The paper mentions using "publicly available Python packages" for baseline methods (e.g., GPy Opt, pysmac, hyperopt) but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | Co Ca BO and one-hot BO both use the UCB acquisition function (Srinivas et al., 2010) with scale parameter κ = 2.0. In all experiments, we tested four different λ values for our method4: λ = 1.0, 0.5, 0.0, auto, where λ = auto means λ is optimised as a hyperparameter. We used a Matérn, ν = 5 2, kernel for kx, as well as for One-hot BO, and used the indicator-based kernel discussed in Section 4.2 for kh. For both our method and One-hot BO, we optimised the GP hyperparameters by maximising the log marginal likelihood every 10 iterations using multi-started gradient descent (see Appendix E for more details). |