Sub-linear Regret Bounds for Bayesian Optimisation in Unknown Search Spaces
Authors: Hung Tran-The, Sunil Gupta, Santu Rana, Huong Ha, Svetha Venkatesh
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments with synthetic and real-world optimisation tasks demonstrate the superiority of our algorithms over the current state-of-the-art methods for Bayesian optimisation in unknown search space. We evaluate our algorithms extensively using a variety of optimisation tasks including optimisation of several benchmark functions and tuning both the hyperparameters (Elastic Net) and parameters of machine learning algorithms (weights of a neural network and Lunar Lander). |
| Researcher Affiliation | Academia | Hung Tran-The , Sunil Gupta, Santu Rana, Huong Ha, Svetha Venkatesh Applied Artificial Intelligence Institute Deakin University, Australia |
| Pseudocode | Yes | Algorithm 1 Hu BO Algorithm Parameters: α Rrate of expanding the volume of the search space Initialisation: Define an initial search space X0 = [a, b]d, a finite domain Cinitial = [cmin, cmax]d, where X0 Cinitial. Sample initial points in X0 to build D0. 1: for t = 1, 2, ...T do 2: Fit the Gaussian process using Dt 1. 3: Define Xt using (2). 4: Find xt = argmaxx Xtut(x), where ut(x) defined as in Eq (1) to find xt. 5: Sample yt = f(xt) + ϵt. 6: Augment the data Dt = {Dt 1, (xt, yt)}. 7: end for |
| Open Source Code | Yes | Our source code is publicly available at https://github.com/Tran-The Hung/Unbounded_Bayesian_Optimisation. |
| Open Datasets | Yes | We train the model using the MNIST train dataset and then evaluate the model using the MNIST test dataset. |
| Dataset Splits | No | The paper mentions 'validation loss' for evaluation but does not provide specific percentages or counts for training, validation, or test dataset splits. |
| Hardware Specification | No | The paper does not mention any specific hardware (e.g., GPU, CPU models, or cloud computing specifications) used for running the experiments. |
| Software Dependencies | No | We implement the Elastic net model by using the function SGDClassifier in the scikit-learn package [20]. For all algorithms, the Squared Exponential kernel is used to model GP. The GP models are fitted using the Maximum Likelihood Estimation. - No version numbers are provided for scikit-learn or other software. |
| Experiment Setup | Yes | Following the setting of the initial search space X0 as in all baselines [23, 18, 8], we select the X0 as 20% of the pre-defined function domain. For example, if X = [0, 1]d, the size of X0 is the 0.2 where its center is placed randomly in the domain [0, 1]d. For our algorithms, we set Cinital as 10 times to the size of X0 along each dimension. ...The function evaluation budget is set to 30d in low dimensions and 10d in high dimensions where d is the input dimension. ... For our algorithms, we set α = 1. ... We use λ = 1, N0 = 1 and thus Nt = t. ... We set the size of hypercubes, lh as 10% of X0. |