Differentiable Neural Architecture Search in Equivalent Space with Exploration Enhancement
Authors: Miao Zhang, Huiqi Li, Shirui Pan, Xiaojun Chang, Zongyuan Ge, Steven Su
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We compared the proposed approach with different One Shot NAS baselines on the NAS benchmark dataset NAS-Bench-201 [13], and extensive experimental results illustrate the effectiveness of our method, which outperforms all baselines on this dataset. and 4 Experiments The high computational cost of evaluating architectures is the major obstacle of analyzing and reproducing One-Shot NAS methods, and it is hard to reproduce current NAS methods under the same experimental setting for a fair comparison. In this section, we adopt the NAS-Bench-201 [13] as the benchmark dataset to analyze our E2NAS. |
| Researcher Affiliation | Collaboration | Miao Zhang1,2,3, Huiqi Li1 , Shirui Pan3 , Xiaojun Chang3, Zongyuan Ge3,4, Steven Su2 1Beijing Institute of Technology 2University of Technology Sydney 3Monash University 4Airdoc Research Australia |
| Pseudocode | Yes | Algorithm 1 E2NAS 1: Input: Trained encoder E and decoder D, training dataset Dtrain and validation dataset Dvalid. 2: Initial architecture archive A = ; 3: Randomly initialize architecture parameter αθ and supernet weights WA(α); 4: while not done do 5: Sample batch of Dtrain, decode αθ to get α based on D, get the complementary architecture αc, and update the supernet weights WA(α) based on Eq. (7), and add architecture α into A; 6: Sample batch of Dvalid, and update αθ based on Eq. (3); 7: end while 8: Decode αθ to obtain the best α based on the trained decoder D. 9: Retrain α and get the best performance on the test dataset Dtest. 10: Return: architecture α with best performance. |
| Open Source Code | No | The paper does not contain any explicit statement about releasing source code or a link to a repository for the described methodology. |
| Open Datasets | Yes | In this section, we adopt the NAS-Bench-201 [13] as the benchmark dataset to analyze our E2NAS. and The search space in NAS-Bench-201 contains four nodes with five associated operations, resulting in 15625 cell candidates. |
| Dataset Splits | Yes | In this section, we adopt the NAS-Bench-201 [13] as the benchmark dataset to analyze our E2NAS. and Method CIFAR-10 CIFAR-100 Image Net-16-120 Valid(%) Test(%) Valid(%) Test(%) Valid(%) Test(%) |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or cloud instance types) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers, such as library names or exact programming language versions. |
| Experiment Setup | Yes | The hyperparameters of E2NAS are set as ε=0.5 and γ=Sigγ(10) in this experiment. and Generally, there are only two additional parameters that need to be specified, ε for the supernet training and λ for architecture parameter learning, in our E2NAS. and We set eight different settings for γ, including four static settings and four Sigmoid-type settings, as shown in the third block of Table 2. |