Cascaded Algorithm-Selection and Hyper-Parameter Optimization with Extreme-Region Upper Confidence Bound Bandit
Authors: Yi-Qi Hu, Yang Yu, Jun-Da Liao
IJCAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also conduct experiments on a synthetic problem as well as a set of Auto ML tasks. The results verify the effectiveness of the proposed method. The experiments on synthetic and real Auto ML tasks reveal that the ER-UCB can find the best algorithm precisely, and exploit it with the majority of the trial budget. |
| Researcher Affiliation | Academia | Yi-Qi Hu , Yang Yu and Jun-Da Liao National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China {huyq, yuy}@lamda.nju.edu.cn, liaojd98@gmail.com |
| Pseudocode | Yes | Algorithm 1 Extreme-region UCB Bandit |
| Open Source Code | No | No explicit statement or link regarding open-source code availability for the methodology described in this paper was found. |
| Open Datasets | Yes | And 12 classification datasets from UCI are selected as Auto ML tasks. |
| Dataset Splits | No | Let Dtrain and Dtest denote the training and testing datasets... The Auto ML problem can be formulated as follows: ... where Dvalid j Dtrain and Dtrain j = Dtrain Dvalid j. (While validation data is mentioned, specific split percentages or methodology for creating the splits from the datasets are not provided.) |
| Hardware Specification | No | No explicit hardware specifications (e.g., specific CPU/GPU models or memory) used for running experiments were provided in the paper. |
| Software Dependencies | No | We select 10 frequently-used algorithms as the candidates from SKLEARN [Pedregosa et al., 2011] (Mentions SKLEARN but no version number.) |
| Experiment Setup | Yes | The trial budget is set as 1000. The experiment for every hyperparameter setting is repeated for 3 times independently, and the average results are presented. The trial budget is 1000. We set θ = 0.01, γ = 20 for the ER-UCB on all datasets. The β is set according to the tasks, and showed in Table 2. For each method and each dataset, we run every experiment 3 times independently, and the average performances of our experiment are presented. |