Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Bayesian Optimistic Optimisation with Exponentially Decaying Regret
Authors: Hung Tran-The, Sunil Gupta, Santu Rana, Svetha Venkatesh
ICML 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform experiments on optimisation of various synthetic functions and machine learning hyperparameter tuning tasks and show that our algorithm outperforms baselines. |
| Researcher Affiliation | Academia | 1Applied Artificial Intelligence Institute, Deakin University, Geelong, Australia. |
| Pseudocode | Yes | Algorithm 1 The BOO Algorithm |
| Open Source Code | No | The paper does not contain any statements or links indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | Using MNIST dataset, we train the models with this hyperparameter setting using the 55000 patterns and then test the model on the 10000 patterns. [...] XGBoost classification We demonstrate a classification task using XGBoost (Chen & Guestrin, 2016) on a Skin Segmentation dataset 1. [...] 1https://archive.ics.uci.edu/ml/datasets/skin+segmentation |
| Dataset Splits | Yes | Using MNIST dataset, we train the models with this hyperparameter setting using the 55000 patterns and then test the model on the 10000 patterns. [...] The Skin Segmentation dataset is plit into 15% for training and 85% for testing for a classification problem. |
| Hardware Specification | No | The paper mentions 'All implementations are in Python' but does not specify any hardware details such as GPU models, CPU types, or memory used for running the experiments. |
| Software Dependencies | No | The paper mentions 'All implementations are in Python' and refers to 'the function SGDClassifier in the scikit-learn package' and 'Adam optimizer', but it does not provide specific version numbers for any of these software dependencies. |
| Experiment Setup | Yes | For each test function, we repeat the experiments 15 times. [...] We used Mat ern kernel with ν = 4+(D+1)/2 which satisfies our assumptions, and estimated the kernel hyper-parameters automatically from data using Maximum Likelihood Estimation. [...] For our BOO algorithm, we choose parameters m, a, b and N as per Corollary 1 which suggests using N so that a = O((N 2 )1/D) 2. For Hartmann3 (D = 3) and Schwefel (D = 3) we use partitioning procedure P(8; 2, 3) with N = 200. For Shekel function (D = 4), we use P(16; 2; 4) with N = 800 so that (N 2 )1/D 2. We follow Lemma 6 in our theoretical analysis to set βp = 2log(π2p3/3η), with η = 0.05. [...] The model is trained with the Adam optimizer in 20 epochs with batch size 128. [...] Table 2. Hyperparameters for XGBoost. |