Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
∇QDARTS: Quantization as an Elastic Dimension to Differentiable NAS
Authors: Payman Behnam, Uday Kamal, Sanjana Vijay Ganesh, Zhaoyi Li, Michael Andrew Jurado, Alind Khare, Igor Fedorov, Gaowen Liu, Alexey Tumanov
TMLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Compared to fp32, QDARTS shows impressive performance on CIFAR10 with (2,4) bit precision, reducing bit operations by 160 with a slight 1.57% accuracy drop. Increasing the capacity enables QDARTS to match fp32 accuracy while reducing bit operations by 18 . For the Image Net dataset, with just (2,4) bit precision, QDARTS outperforms state-of-the-art methods such as APQ, SPOS, OQA, and MNAS by 2.3%, 2.9%, 0.3%, and 2.7% in terms of accuracy. By incorporating (2,4,8) bit precision, QDARTS further minimizes the accuracy drop to 1% compared to fp32, alongside a substantial reduction of 17 in required bit operations and 2.6 in memory footprint. In terms of bit-operation (memory footprint), QDARTS excels over APQ, SPOS, OQA, and MNAS with similar accuracy by 2.3 (12 ), 2.4 (3 ), 13% (6.2 ), 3.4 (37%), for bit-operation (memory footprint), respectively. Table 2 and Table 3 compare accuracy, required bit operations (Bit Ops), and memory footprint of different baselines with QDARTS. The last row in Table 2 shows that QDARTS reaches the same accuracy as PC-DARTS with 18 less Bit Ops and 5 less memory footprint. 4.8 Ablation Study |
| Researcher Affiliation | Collaboration | Payman Behnam* , Uday Kamal*, Sanjana Vijay Ganesh, Zhaoyi Li, Michael Andrew Jurado, Alind Khare Georgia Institute of Technology Igor Fedorov Meta Gaowen Liu Cisco Research Alexey Tumanov Georgia Institute of Technology |
| Pseudocode | No | The paper describes methods using mathematical equations and textual explanations, but it does not contain any explicitly labeled pseudocode blocks or algorithms in a structured, code-like format. |
| Open Source Code | Yes | We have released the source code on Git Hub1. 1https://github.com/gatech-sysml/Nabla QDARTS |
| Open Datasets | Yes | We perform QDARTS experiments on CIFAR10 and Image Net, as the two most popular datasets for evaluating the efficiency and scalability of the NAS algorithm. |
| Dataset Splits | Yes | We perform QDARTS experiments on CIFAR10 and Image Net, as the two most popular datasets for evaluating the efficiency and scalability of the NAS algorithm. |
| Hardware Specification | Yes | The experiments are carried out on 8 (1) A40 GPUs on our internal clusters for Image Net (CIFAR10) datasets. ... For FPGA implementation, we use AMD Xilinx Alveo U250 FPGA (AMD (2023)). |
| Software Dependencies | No | The paper mentions software tools and frameworks such as HWGQ, channel-wise min-max, ffcv-based accelerator, Timeloop, Accelergy, and FINN framework, but it does not provide specific version numbers for these components or other programming libraries. |
| Experiment Setup | Yes | The training batch size is 2048(256) for Image Net (CIFAR10) datasets. We consider 500 epochs for finetuning the searched architecture for all our experiments. During the search phase, we use different optimizers for the ω, α, and γ. For the ω, an SGD optimizer with a learning rate of 0.5 (0.1 for CIFAR10), momentum 0.9, and weight decay of 3 10 4 is used. For the α, Adam optimizer with learning rate 6 10 3 (6 10 4 for CIFAR10), beta1 0.5, beta2 0.999, and weight decay of 10 3 is used. Finally, for γ, an SGD optimizer with a learning rate of 0.01, momentum of 0.9, and weight decay of 10 3 is used. |