Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search
Authors: YAO SHU, Zhongxiang Dai, Zhaoxuan Wu, Bryan Kian Hsiang Low
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our codes are available at https://github.com/shuyao95/HNAS. [...] We use extensive experiments to verify the insights derived from our unified theoretical analysis, as well as the search effectiveness and efficiency of our non-trivial HNAS framework (Sec. 6). |
| Researcher Affiliation | Academia | Dept. of Computer Science, National University of Singapore, Republic of Singapore Institute of Data Science, National University of Singapore, Republic of Singapore Integrative Sciences and Engineering Programme, NUSGS, Republic of Singapore |
| Pseudocode | Yes | Algorithm 1: Hybrid Neural Architecture Search (HNAS) |
| Open Source Code | Yes | Our codes are available at https://github.com/shuyao95/HNAS. |
| Open Datasets | Yes | NAS-Bench-101 [22] and NAS-Bench-201 [23] with CIFAR-10 [24]. |
| Dataset Splits | Yes | We firstly validate the theoretical connections between MTrace and other training-free metrics from Sec. 3.2 by examining their Spearman correlations for all architectures in NAS-Bench-101 [22] and NAS-Bench-201 [23] with CIFAR-10 [24]. |
| Hardware Specification | Yes | The result of HNAS is reported with the mean and standard deviation of 5 independent searches and its search costs are evaluated on a Nvidia 1080Ti. |
| Software Dependencies | No | The paper states that codes are available at a GitHub link, but it does not explicitly list specific software dependencies with version numbers (e.g., Python, PyTorch, CUDA versions) in the provided text. |
| Experiment Setup | Yes | The results in Figure 1 show that MTrace and other training-free metrics from Sec. 3.2 are indeed highly correlated since they consistently achieve high positive correlations in different search spaces. Of note, we will follow the same approach to evaluate these training-free metrics in our following sections. The results in Figure 1 show that all these training-free metrics are evaluated using a batch (with size 64) of sampled data following that of [9]. |