Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Nonmyopic Multifidelity Acitve Search
Authors: Quan Nguyen, Arghavan Modiri, Roman Garnett
ICML 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the performance of our solution on real-world datasets and demonstrate significantly better performance than natural benchmarks. |
| Researcher Affiliation | Academia | 1Washington University in St. Louis, MO, USA 2University of Toronto, Toronto, Canada. |
| Pseudocode | Yes | We give the pseudocode for the policy in the appendix. |
| Open Source Code | Yes | Matlab implementations of our policies are available at: https://github.com/KrisNguyen135/multifidelity-active-search . |
| Open Datasets | Yes | Here we used the first 50 proteins from the Binding DB database (Liu et al., 2007) described by Jiang et al. (2017). A set of 100 000 compounds sampled from the ZINC database (Sterling & Irwin, 2015) served as a shared negative set. ...This dataset comprises 106 810 alloys from the materials literature (Kawazoe et al., 1997; Ward et al., 2016) |
| Dataset Splits | No | The paper does not explicitly provide training/validation/test dataset splits with percentages or sample counts. It describes an iterative active search process within a budget. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper states 'Matlab implementations of our policies', but does not provide specific version numbers for Matlab or any other software dependencies. |
| Experiment Setup | Yes | We set θ {0.1, 0.3}. We set k, the number of L queries that are made for each H query, to be either 2 or 5, and set the budget on H to be 300. ... In our experiments, we set u = s = 500 for MF ENS. ... We set β = 0.01 for L queries and β = 0.001 for H queries, as suggested in the same work. |