EZNAS: Evolving Zero-Cost Proxies For Neural Architecture Scoring
Authors: Yash Akhauri, Juan Munoz, Nilesh Jain, Ravishankar Iyer
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our methodology efficiently discovers an interpretable and generalizable zero-cost proxy that gives state of the art score-accuracy correlation on all datasets and search spaces of NASBench201 and Network Design Spaces (NDS). |
| Researcher Affiliation | Collaboration | Yash Akhauri1 J. Pablo Muñoz2 Nilesh Jain2 Ravi Iyer2 1Cornell University 2Intel Labs |
| Pseudocode | Yes | Algorithm 1 EZNAS Search Algorithm |
| Open Source Code | No | We plan to open-source our work as part of a larger framework in the future. |
| Open Datasets | Yes | We test the fittest program from our final population as well as the two fittest programs encountered through-out the evolutionary search. At test time, we take the program and find the score-accuracy correlation over 4000 neural network architectures sampled from the NASBench-201 and NDS design spaces. |
| Dataset Splits | Yes | To address over-fitting of programs to small datasets of network statistics while minimizing compute resources required for evaluating on the entire dataset of network statistics, we generate an evolution task dataset. This is generated by randomly sampling 80 neural networks from each available search space (NASBench-201 and NDS) and dataset (CIFAR-10, CIFAR-100, Image Net-16-120). |
| Hardware Specification | Yes | Each search takes approximately 24 hours on a Intel(R) Xeon(R) Gold 6242 CPU with 1 terabyte of RAM. |
| Software Dependencies | No | In our tests, we utilize the VarOr implementation from Distributed Evolutionary Algorithms in Python (DEAP) (Fortin et al. [2012]) framework for the variation of individual programs. The specific version number of DEAP or Python is not provided. |
| Experiment Setup | Yes | We have placed static limits on the depth of all expression trees at 10. We consider s as a hyper-parameter. In our tests, this is consistently kept at 4. |