Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Revolutionizing Training-Free NAS: Towards Efficient Automatic Proxy Discovery via Large Language Models
Authors: Haidong Kang, Lihong Lin, Hanling Wang
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on mainstream NAS benchmarks, demonstrating APD excels in both performance and efficiency. Besides, we firmly believe that our APD will dramatically benefit the deep learning community through providing novel paradigm of design algorithms via LLMs. |
| Researcher Affiliation | Academia | 1Northeastern University, China 2Pengcheng Laboratory, China |
| Pseudocode | Yes | Algorithm 1 Evolution Framework |
| Open Source Code | Yes | Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We provided runnable code in the supplementary material. |
| Open Datasets | Yes | We evaluate APD on 5 representative search spaces (e.g., NAS-Bench-201 Dong and Yang [2020], NAS-Bench-101 Ying et al. [2019], DARTS Liu et al. [2018], Trans NAS-Bench-101 Micro Duan et al. [2021], Oo D-Vi T-NAS Bai et al. [2021]) across 4 tasks: image recognition, autoencoding, scene classification, and self-supervised jigsaw puzzles. |
| Dataset Splits | Yes | We evaluate APD on 5 representative search spaces (e.g., NAS-Bench-201 Dong and Yang [2020], NAS-Bench-101 Ying et al. [2019], DARTS Liu et al. [2018], Trans NAS-Bench-101 Micro Duan et al. [2021], Oo D-Vi T-NAS Bai et al. [2021]) across 4 tasks: image recognition, autoencoding, scene classification, and self-supervised jigsaw puzzles. We transfer the zero-cost proxy identified on CIFAR-10 within the NAS-Bench-201 search space to all datasets in both NAS-Bench-201 and NAS-Bench-101. |
| Hardware Specification | Yes | Searches are conducted on a single RTX4090 GPU with a fixed random seed of 0. |
| Software Dependencies | No | Output: For each proxy, provide a Description paragraph and Code demo implementing evaluate(model, inputs, targets) in Py Torch. |
| Experiment Setup | Yes | Table 8: Hyper-parameters for Automatic Proxy Discovery. Parameter Value Episodes 10 Steps 100 History windows 5 Discount factor 0.9 Actor learning rate 1e-3 Critic learning rate 1e-2 Input batch size 16 Repeats 5 Hidden size 256 Number of layers 2 Population size 5 β 1 |