Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
A General Recipe for Likelihood-free Bayesian Optimization
Authors: Jiaming Song, Lantao Yu, Willie Neiswanger, Stefano Ermon
ICML 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our LFBO method empirically on a number of synthetic and real optimization problems. On several real-world benchmarks, LFBO outperforms various state-of-the-art methods in black-box optimization. |
| Researcher Affiliation | Collaboration | 1NVIDIA (Work done while at Stanford). 2Stanford University. Correspondence to: Jiaming Song <EMAIL>. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Project website: https://lfbo-ml.github.io/ |
| Open Datasets | Yes | In HPOBench, we aim to find the optimal hyperparameter configurations for training a two-layer feed-forward neural networks on four popular UCI datasets (Asuncion & Newman, 2007) for regression... In NAS-Bench-201... three datasets: CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009) and Image Net-16 (Chrabaszcz et al., 2017). |
| Dataset Splits | No | The paper describes using tabular benchmarks (HPOBench, NAS-Bench-201) where configurations and their results are pre-tabulated. It does not specify explicit training/validation/test splits that they performed for their black-box optimization process. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions software like Keras, scikit-learn, and XGBoost, and provides links to their websites, but does not specify exact version numbers for these dependencies. |
| Experiment Setup | Yes | Our classifier model is a two layer fully-connected neural network with 128 units at each layer... For each evaluation, we optimize the model with batch gradient descent for 1000 epochs, using an Adam optimizer with learning rate 0.01 and weight decay 10 6. |