Batch Bayesian Optimization For Replicable Experimental Design
Authors: Zhongxiang Dai, Quoc Phong Nguyen, Sebastian Tay, Daisuke Urano, Richalynn Leong, Bryan Kian Hsiang Low, Patrick Jaillet
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also show the effectiveness of our algorithms in two practical real-world applications: precision agriculture and Auto ML. (Abstract, p. 1) and In addition to our theoretical contributions, we also demonstrate the practical efficacy of our algorithms in two real-world problems (Sec. 5). (Section 1, p. 2) |
| Researcher Affiliation | Academia | 1Department of Computer Science, National University of Singapore 2LIDS and 3EECS, Massachusetts Institute of Technology 4Institute for Infocomm Research (I2R), A*STAR, Singapore 5Temasek Life Sciences Laboratory, Singapore |
| Pseudocode | Yes | Algorithm 1 BTS-RED-Known. (Section 3.1.1, p. 3) and Algorithm 2 Mean-Var-BTS-RED. (Section 4, p. 6) and Algorithm 3 BTS-RED-Unknown. (Appendix F, p. 16) |
| Open Source Code | No | The paper does not contain an explicit statement about releasing the source code for its methodology or a direct link to a repository containing it. |
| Open Datasets | Yes | We adopt the EMNIST dataset which is widely used in multi-task learning [10, 15]. (Section 5.3, p. 9) and The EMNIST dataset is under the CC0 license. (Appendix H.3, p. 19) |
| Dataset Splits | No | The paper describes data collection (e.g., replicating plant conditions 6 times, evaluating EMNIST on 100 tasks to construct groundtruth functions) but does not provide specific training/validation/test dataset splits (e.g., percentages, sample counts, or predefined splits) for model training. |
| Hardware Specification | Yes | Our experiments are run on a computer server with 128 CPUs, with the AMD EPYC 7543 32-Core Processor. The server has 8 NVIDIA Ge Force RTX 3080 GPUs. (Appendix H, p. 17) |
| Software Dependencies | No | The paper mentions using 'GPflow' and implies other standard machine learning libraries (e.g., for SVMs), but it does not specify version numbers for these software dependencies, which are necessary for full reproducibility. |
| Experiment Setup | Yes | For BTS-RED-Known and BTS-RED-Unknown...we set nmax = B/2 in the first T/2 iterations and nmax = B subsequently...We set nmin = 2 unless specified otherwise...We set B = 50...We choose the effective noise variance R2 by following our theoretical guideline in Sec. 3.1.2, i.e., R2 = κσ2 max( B + 1)/(B 1)...We only use two values of κ = 0.2 and κ = 0.3 in all experiments. (Section 5, p. 7) |