Neural Architecture Search without Training
Authors: Joe Mellor, Jack Turner, Amos Storkey, Elliot J Crowley
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We incorporate this measure into a simple algorithm that allows us to search for powerful networks without any training in a matter of seconds on a single GPU, and verify its effectiveness on NAS-Bench-101, NASBench-201, NATS-Bench, and Network Design Spaces. |
| Researcher Affiliation | Academia | 1Usher Institute, University of Edinburgh 2School of Informatics, University of Edinburgh 3School of Engineering, University of Edinburgh. Correspondence to: Joseph Mellor <joe.mellor@ed.ac.uk>. |
| Pseudocode | Yes | Algorithm 1 NASWOT; Algorithm 2 Assisted Regularised EA AREA |
| Open Source Code | Yes | Code for reproducing our experiments is available at https://github.com/ Bayes Watch/nas-without-training. |
| Open Datasets | Yes | In this work we utilise NAS-Bench-101 (Ying et al., 2019), NASBench-201 (Dong & Yang, 2020), and NATS-Bench (Dong et al., 2021) to evaluate the effectiveness of our approach. NAS-Bench-101 consists of 423,624 neural networks... on the CIFAR-10 dataset... NAS-Bench-201 consists of 15,625 networks trained multiple times on CIFAR-10, CIFAR-100, and Image Net-16-120 (Chrabaszcz et al., 2017). We also make use of the Network Design Spaces (NDS) dataset (Radosavovic et al., 2019). |
| Dataset Splits | Yes | NAS-Bench-101 consists of 423,624 neural networks that have been trained exhaustively, with three different initialisations, on the CIFAR-10 dataset for 108 epochs. NAS-Bench-201 consists of 15,625 networks trained multiple times on CIFAR-10, CIFAR-100, and Image Net-16-120. In this work we utilise NAS-Bench-101 (Ying et al., 2019), NASBench-201 (Dong & Yang, 2020), and NATS-Bench (Dong et al., 2021) to evaluate the effectiveness of our approach. We sample networks at random and plot our score s on the untrained networks against their validation accuracies when trained. |
| Hardware Specification | Yes | Search times are recorded for a single 1080Ti GPU. |
| Software Dependencies | No | The paper mentions using NAS-Bench APIs, but it does not specify version numbers for any software dependencies like Python, PyTorch, or other libraries. |
| Experiment Setup | Yes | NASWOT is our training-free algorithm (across 500 runs). REA uses evolutionary search to select an architecture (50 runs), Random selects one architecture (500 runs). AREA (assisted-REA) uses our score (Equation 2) to select the starting population for REA (50 runs). Search times for REA and AREA were calculated using the NASBench-101 API. We report NASWOT for sample sizes of N=10, N=100, and N=1000. |