Meta-Learning Reliable Priors in the Function Space
Authors: Jonas Rothfuss, Dominique Heyn, jinfan Chen, Andreas Krause
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our benchmark study on meta-learning for Bayesian Optimization (BO), F-PACOH significantly outperforms all other meta-learners and standard baselines. |
| Researcher Affiliation | Academia | Jonas Rothfuss ETH Zurich jonas.rothfuss@inf.ethz.ch Dominique Heyn ETH Zurich heynd@student.ethz.ch Jinfan Chen ETH Zurich georgcjf@gmail.com Andreas Krause ETH Zurich krausea@ethz.ch |
| Pseudocode | Yes | Algorithm 1 F-PACOH-MAP: Meta-Learning Reliable Priors |
| Open Source Code | Yes | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] See Appendix B.3. |
| Open Datasets | Yes | As practical application of meta-learning for BO, we consider hyper-parameter tuning of machine learning algorithms on different datasets... [67]. In these hyper-parameter experiments, we use the area under the ROC curve (AUROC) as test metric to optimize. |
| Dataset Splits | Yes | With this meta-training data D1,Ti, ..., D1,Tn, we meta-train F-PACOH and the considered baselines. To obtain statistically robust results, we perform independent BO runs on 10 unseen target functions / tasks and we repeat the whole meta-training & -testing process for 25 random seeds to initialize the meta-learner. |
| Hardware Specification | Yes | All experiments were performed on a cluster of Intel (R) Xeon (R) CPU E5-2698 v4 @ 2.20GHz machines with 40 CPU cores and 192 GB RAM. |
| Software Dependencies | No | The paper mentions software like 'GPyTorch [70]' and 'Adam optimizer [71]' but does not provide specific version numbers for these or other key software components, such as PyTorch itself, which would be needed for reproducible results. |
| Experiment Setup | Yes | Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix B |