CapsAndRuns: An Improved Method for Approximately Optimal Algorithm Configuration
Authors: Gellert Weisz, Andras Gyorgy, Csaba Szepesvari
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments verify that our method can significantly outperform its competitors. |
| Researcher Affiliation | Collaboration | 1Deep Mind, London, UK. 2On leave from Imperial College London, London, UK. 3On leave from University of Alberta, Edmonton, AB, Canada. |
| Pseudocode | Yes | Algorithm 1 CAPSANDRUNS, Algorithm 2 QUANTILEEST, Algorithm 3 RUNTIMEEST |
| Open Source Code | No | The paper mentions using the "open-source minisat solver" but does not state that the code for CAPSANDRUNS or their experiments is open-source or provide a link to it. |
| Open Datasets | Yes | In the experiments we used the same benchmark dataset as in our previous paper (Weisz et al., 2018). The dataset contains runtimes of 972 different configurations of minisat on a set of 20118 SAT problems generated using CNFuzz DD,7 with a timeout of 15 CPU minutes. 7http://fmv.jku.at/cnfuzzdd/ |
| Dataset Splits | No | The paper mentions using a set of 20118 SAT problems but does not provide specific details on how this dataset was split into training, validation, or test sets. |
| Hardware Specification | No | The paper does not provide any specific hardware details (e.g., CPU, GPU models, memory) used for running the experiments or simulations. |
| Software Dependencies | No | The paper mentions various tools and solvers like "minisat solver (Sorensson & Een, 2005)", "SMAC (Hutter et al., 2011; 2013)", "Param ILS (Hutter, 2007; Hutter et al., 2009)", "GGA (Ans otegui et al., 2009; 2015)", and "irace (Birattari et al., 2002; L opez-Ib anez et al., 2011)", but it does not specify version numbers for these or other software dependencies. |
| Experiment Setup | Yes | We simulated runs of STRUCTURED PROCRASTINATION (Kleinberg et al., 2017), LEAPSANDBOUNDS (Weisz et al., 2018), and CAPSANDRUNS (this work), with parameters ε = 0.05, δ = 0.2, and error probability 0.1 (ζ = 1/60) |