CAP: A Context-Aware Neural Predictor for NAS
Authors: Han Ji, Yuqi Feng, Yanan Sun
IJCAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results in different search spaces demonstrate the superior performance of CAP compared with state-of-the-art neural predictors. In particular, CAP can rank architectures precisely at the budget of only 172 annotated architectures in NAS-Bench101. Moreover, CAP can help find promising architectures in both NAS-Bench-101 and DARTS search spaces on the CIFAR-10 dataset, serving as a useful navigator for NAS to explore the search space efficiently. |
| Researcher Affiliation | Academia | Han Ji , Yuqi Feng and Yanan Sun College of Computer Science, Sichuan University jihan@stu.scu.edu.cn, feng770623@gmail.com, ysun@scu.edu.cn |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain an explicit statement about the release of its own source code, nor does it provide a link to a repository. |
| Open Datasets | Yes | NAS-Bench-101 Search Space. The NAS-Bench-101 is a cell-based search space consisting of over 423k unique neural architectures. In each architecture cell, at most seven nodes and nine edges are contained. The nodes represent operation layers and the edges represent connection manners. The search space provides the validation accuracy and test accuracy of convolutional neural networks (CNNs) on CIFAR-10. NAS-Bench-201 Search Space. The NAS-Bench-201 is also a cell-based search space constructed by over 15K different neural architectures. ... Detailed results of CNNs are available on three datasets: CIFAR-10, CIFAR-100, and Image Net16-120. DARTS Search Space. The DARTS search space is built by both normal cells and reduction cells where each cell is composed of seven nodes and 14 edges. It is a much larger open domain search space than the NAS-Bench-101 and the NAS-Bench-201 search spaces. |
| Dataset Splits | Yes | Train Samples 100 172 424 424 4236 Validation Samples 200 200 200 200 200 Test Samples all all 100 all all |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | Specifically, 50 annotated architectures are randomly chosen to train the predictor, and the predictor afterwards evaluates all the architectures in the search space. The best performance of the top 50 evaluated architectures is reported as the search result. For each candidate architecture, the epoch number is set to 50 with a batch size of 96. As for the searching method, we sample 100k architectures at random in the DARTS search space and they are afterwards evaluated by our trained predictor. After that, we select the architectures with top-3 predicted performance as the search results and re-train them with common DARTS strategies. We choose a two-layer MLP as the regression model. Here, we use Bayesian personalized ranking (BPR) [Rendle et al., 2012] loss for the sake of its popularity in relative ranking tasks. The optimal number L is three and deeper GIN leads to the decrease in predictor performance. We find that the ranking performance is satisfactory when the central subgraph in each architecture and the corresponding context graphs share only a few overlapping nodes (i.e. K = 1, R = 2). The negative sampling ratio is set to one... |