Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Cascaded Algorithm-Selection and Hyper-Parameter Optimization with Extreme-Region Upper Confidence Bound Bandit
Authors: Yi-Qi Hu, Yang Yu, Jun-Da Liao
IJCAI 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also conduct experiments on a synthetic problem as well as a set of Auto ML tasks. The results verify the effectiveness of the proposed method. The experiments on synthetic and real Auto ML tasks reveal that the ER-UCB can find the best algorithm precisely, and exploit it with the majority of the trial budget. |
| Researcher Affiliation | Academia | Yi-Qi Hu , Yang Yu and Jun-Da Liao National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1 Extreme-region UCB Bandit |
| Open Source Code | No | No explicit statement or link regarding open-source code availability for the methodology described in this paper was found. |
| Open Datasets | Yes | And 12 classification datasets from UCI are selected as Auto ML tasks. |
| Dataset Splits | No | Let Dtrain and Dtest denote the training and testing datasets... The Auto ML problem can be formulated as follows: ... where Dvalid j Dtrain and Dtrain j = Dtrain Dvalid j. (While validation data is mentioned, specific split percentages or methodology for creating the splits from the datasets are not provided.) |
| Hardware Specification | No | No explicit hardware specifications (e.g., specific CPU/GPU models or memory) used for running experiments were provided in the paper. |
| Software Dependencies | No | We select 10 frequently-used algorithms as the candidates from SKLEARN [Pedregosa et al., 2011] (Mentions SKLEARN but no version number.) |
| Experiment Setup | Yes | The trial budget is set as 1000. The experiment for every hyperparameter setting is repeated for 3 times independently, and the average results are presented. The trial budget is 1000. We set θ = 0.01, γ = 20 for the ER-UCB on all datasets. The β is set according to the tasks, and showed in Table 2. For each method and each dataset, we run every experiment 3 times independently, and the average performances of our experiment are presented. |