MathNAS: If Blocks Have a Role in Mathematical Architecture Design
Authors: Qinsi Wang, Jinghan Ke, Zhi Liang, Sihai Zhang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The superiority of Math NAS is validated on multiple large-scale CV and NLP benchmark datasets. Notably on Image Net-1k, Math NAS achieves 82.5% top-1 accuracy, 1.2% and 0.96% higher than Swin-T and Le Vi T-256, respectively. |
| Researcher Affiliation | Academia | 1University of Science and Technology of China 2School of Life Sciences, University of Science and Technology of China 3Key Laboratory of Wireless-Optical Communications, Chinese Academy of Sciences 4School of Microelectronics, University of Science and Technology of China |
| Pseudocode | Yes | Algorithm 1 Math Neural Architecture Search |
| Open Source Code | Yes | Our code is available at https://github.com/wangqinsi1/Math NAS. |
| Open Datasets | Yes | The superiority of Math NAS is validated on multiple large-scale CV and NLP benchmark datasets. Notably on Image Net-1k, Math NAS achieves 82.5% top-1 accuracy... |
| Dataset Splits | Yes | For NAS-Bench-201 and Mobile Net V3, we adopt the training methodology employed in [16] and [22] to train the base net for 100 epochs. ... We sample 5 blocks per block node and count the accuracies of 3125 subnetworks on the Image Net validation set. |
| Hardware Specification | Yes | Raspberry Pi Intel Xeon CPU Nvidia TITAN Xp GPU |
| Software Dependencies | No | We employ the Gurobipy solver to address the ILP problem. No version number is specified for Gurobipy or any other software. |
| Experiment Setup | No | The settings of hyperparameters in the training are consistent with the original paper. No specific hyperparameter values (e.g., learning rate, batch size, optimizer settings) or explicit training configuration details are provided within this paper's main text. |