Bayesian optimization for automated model selection
Authors: Gustavo Malkomes, Charles Schaff, Roman Garnett
NeurIPS 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Here we evaluate our proposed algorithm. We split our evaluation into two parts: first, we show that our GP model for predicting a model s evidence is suitable; we then demonstrate that our model search method quickly finds a good model for a range of regression datasets. The datasets we consider are publicly available4 and were used in previous related work [1, 3]. AIRLINE, MAUNA LOA, METHANE, and SOLAR are 1d time series, and CONCRETE and HOUSING have, respectively, 8 and 13 dimensions. Table 1: Root mean square error for model-evidence regression experiment. |
| Researcher Affiliation | Academia | Gustavo Malkomes, Chip Schaff, Roman Garnett Department of Computer Science and Engineering Washington University in St. Louis St. Louis, MO 63130 {luizgustavo, cbschaff, garnett}@wustl.edu |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code or links to a code repository. |
| Open Datasets | Yes | The datasets we consider are publicly available4 and were used in previous related work [1, 3]. AIRLINE, MAUNA LOA, METHANE, and SOLAR are 1d time series, and CONCRETE and HOUSING have, respectively, 8 and 13 dimensions. 4https://archive.ics.uci.edu/ml/datasets.html |
| Dataset Splits | No | The paper states 'We train several baselines on a subset of Dg and test their ability to predict the evidence of the remaining models,' but does not provide specific training, validation, or test split percentages or exact counts for the datasets. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'L-BFGS to optimize model hyperparameters' but does not provide specific versions for any software or libraries used in the experiments. |
| Experiment Setup | Yes | We used L-BFGS to optimize model hyperparameters, using multiple restarts to avoid bad local maxima; each restart begins from a sample from p(θ | M). For BOMS, we always began our search evaluating SE first. The active set of models C ( 4.5) was initialized with all models that are at most two edges distant from the base kernels. To avoid unnecessary re-training over g, we optimized the hyperparameters of µg and Kg every 10 iterations. |