Rethink DARTS Search Space and Renovate a New Benchmark
Authors: Jiuling Zhang, Zhiming Ding
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We re-implement twelve baselines and evaluate them across twelve conditions by combining two underexpolored influential factors: transductive robustness and discretization policy, to reasonably construct a benchmark upon multi-condition evaluation. |
| Researcher Affiliation | Academia | 1University of Chinese Academy of Sciences, Beijing, China 2Institute of Software, Chinese Academy of Sciences, Beijing, China. |
| Pseudocode | No | The paper does not contain any sections explicitly labeled 'Pseudocode' or 'Algorithm', nor does it present any structured, code-like blocks describing a procedure. |
| Open Source Code | Yes | https://github.com/chaoji90/LHD |
| Open Datasets | Yes | Our benchmark is evaluated on the most commonly used standard benchmark datasets (CIFAR-10, CIFAR-100, SVHN) |
| Dataset Splits | Yes | architecture parameters α and operation weights ω are alternately optimized on validation set and training set respectively through a bilevel optimization objective. |
| Hardware Specification | Yes | 5.2h 3.1h on RTX 3090, like-for-like comparison after aligning all other conditions). |
| Software Dependencies | Yes | Software version for search and evaluation of the benchmark: torch 1.9, cuda 11.1, cudnn 8.2, driver version 460.67. But we also test the search and evaluation codes and verify the empirical memory overhead on more recent version: torch 1.10, cuda 11.3, cudnn 8.3 and driver 495.44. |
| Experiment Setup | Yes | Table 8. Hyperparameter settings of baselines in search. |