Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
ImpatientCapsAndRuns: Approximately Optimal Algorithm Configuration from an Infinite Pool
Authors: Gellert Weisz, András György, Wei-I Lin, Devon Graham, Kevin Leyton-Brown, Csaba Szepesvari, Brendan Lucier
NeurIPS 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results demonstrate a practical improvement. |
| Researcher Affiliation | Collaboration | Gellért Weisz Deepmind/University College London, UK András György Deepmind, UK Wei-I Lin University of British Columbia, Canada Devon Graham University of British Columbia, Canada Kevin Leyton-Brown University of British Columbia, Canada Csaba Szepesvári Deepmind/University of Alberta, Canada Brendan Lucier Microsoft Research, USA |
| Pseudocode | Yes | Global variables 1: Instance distribution Γ 2: Phase I measurements count b 3: T 1 . Upper bound on OPTγδ/2, updated continuously by all parallel processes 4: Set N of algorithm configurations Algorithm 1 IMPATIENTCAPSANDRUNS... Algorithm 2 CAPSANDRUNS thread... Algorithm 3 QUANTILEEST... Algorithm 4 PRECHECK... Algorithm 5 RUNTIMEEST |
| Open Source Code | No | The paper does not contain any explicit statement about providing open-source code for the described methodology, nor does it provide a direct link to a code repository for its implementation. |
| Open Datasets | Yes | We looked at two datasets from MIP and one from SAT. We considered true runtime data from the minisat SAT solver on instances generated by CNFuzz DD (http://fmv.jku.at/ cnfuzzdd)... For the MIP scenarios, we looked at the CPLEX integer program solver on combinatorial auction instances (Regions200 [27]) and problems from wildlife conservation (RCW [1]). |
| Dataset Splits | No | The paper does not specify how the datasets were split into training, validation, or test sets for their experiments, nor does it reference standard predefined splits with sufficient detail. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments (e.g., CPU/GPU models, memory specifications, or cloud instance types). |
| Software Dependencies | No | The paper mentions software like 'minisat SAT solver' and 'CPLEX integer program solver' and 'random forest model' but does not provide specific version numbers for these or other software dependencies required for replication. |
| Experiment Setup | No | The paper does not provide specific experimental setup details such as hyperparameters (e.g., learning rates, batch sizes), optimizer settings, or detailed training configurations for the described algorithm or models beyond its own input parameters like epsilon, delta, and gamma which are discussed in relation to the algorithm's guarantees. |