Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Bayesian Adaptive Calibration and Optimal Design
Authors: Rafael Oliveira, Dino Sejdinovic, David Howard, Edwin V. Bonilla
NeurIPS 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show the benefits of our method when compared to related approaches across synthetic and real-data problems. |
| Researcher Affiliation | Collaboration | Rafael Oliveira CSIRO s Data61 Sydney, Australia Dino Sejdinovic University of Adelaide Adelaide, Australia David Howard CSIRO s Data61 Brisbane, Australia Edwin V. Bonilla CSIRO s Data61 Sydney, Australia |
| Pseudocode | Yes | Algorithm 1 BACON |
| Open Source Code | Yes | 4Code available at: https://github.com/csiro-funml/bacon |
| Open Datasets | No | For this experiment, we are provided with a dataset containing R = 10 real measurements of the peak grasping force of soft robotic gripper designs on a range of testing objects (see Fig. 3). |
| Dataset Splits | No | The paper mentions initial data and test points for evaluation, but does not explicitly describe a distinct validation dataset split used for hyperparameter tuning or model selection during training. |
| Hardware Specification | No | The paper mentions that experiments run on a 'high-performance computing platform' and 'CSIRO IMT Scientific Computing' but does not provide specific hardware details like GPU/CPU models or memory. |
| Software Dependencies | No | Our implementation for BACON and most of the baselines, except for VBMC,6 is based on Pyro probabilistic programming models [50]. Gaussian process modelling code is based on Bo Torch7 [51]. The flow architecture is chosen for each synthetic-data problem by running hyper-parameter tuning with a simplified version of the problem. Most Gaussian process models are parameterised with Mat ern kernels [2, Ch. 4] and constant or zero mean functions. Pyro s MCMC with its default no-U-turn (NUTS) sampler [52] was applied to obtain samples from p(θ |Dt 1) at each iteration t. KL divergences are computed from samples using a nearest neighbours estimator implemented in the information theoretical estimators (ITE) package8 [41]. |
| Experiment Setup | Yes | We run each algorithm for T := 50 iterations using a batch of B := 4 designs per iteration. Each of the methods using GP approximations for the simulator are initialised with 20 observations and R = 5 real data points. ... Gradient-based optimisation is run using Adam with a learning rate 10 3 for the flow parameters and 0.05 for the simulation design points, both using cosine annealing with warm restarts as a learning rate scheduler. |