Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Bayesian Optimization for Iterative Learning
Authors: Vu Nguyen, Sebastian Schulze, Michael Osborne
NeurIPS 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the efficiency of our algorithm by tuning hyperparameters for the training of deep reinforcement learning agents and convolutional neural networks. Our algorithm outperforms all existing baselines in identifying optimal hyperparameters in minimal time. |
| Researcher Affiliation | Academia | Vu Nguyen University of Oxford EMAIL Sebastian Schulze University of Oxford EMAIL Michael A. Osborne University of Oxford EMAIL |
| Pseudocode | Yes | Algorithm 1 Bayesian Optimization with Iterative Learning (BOIL) |
| Open Source Code | Yes | We release our implementation at https://github.com/ntienvu/BOIL. |
| Open Datasets | Yes | We consider three DRL settings including a Dueling DQN (DDQN) [46] agent in the Cart Pole-v0 environment and Advantage Actor Critic (A2C) [25] agents in the Inverted Pendulum-v2 and Reacher-v2 environments. In addition to the DRL applications, we tune 6 hyperparameters for training a convolutional neural network [21] on the SVHN dataset and CIFAR10. |
| Dataset Splits | No | The paper mentions using specific datasets like Cart Pole-v0, Inverted Pendulum-v2, Reacher-v2, SVHN, and CIFAR10, but it does not explicitly provide details about training, validation, or test splits (e.g., percentages, sample counts, or specific split methodologies) in the main text. |
| Hardware Specification | Yes | All experiments are executed on a NVIDIA 1080 GTX GPU using the tensorflow-gpu Python package. |
| Software Dependencies | No | The paper mentions 'tensorflow-gpu Python package' but does not specify a version number for TensorFlow or any other software dependency, which is required for reproducibility. |
| Experiment Setup | Yes | We set the maximum number of augmented points to be M = 15 and a threshold for a natural log of GP condition number δ = 20. |