One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Training
Authors: Lianbo Ma, Yuee Zhou, Jianlun Ma, Guo Yu, Qing Li
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | A series of theoretical analysis and experiments on benchmark deep models have demonstrated the effectiveness and competitiveness of the proposed method, and our method especially outperforms others on the convergence performance. Experiment Experimental Setup Four widely-used datasets (CIFAR10, MNIST, SVHN and ILSVRC12 (Image Net)) are applied to validate the performance of BLAQ. |
| Researcher Affiliation | Academia | Lianbo Ma1, Yuee Zhou1, Jianlun Ma1, Guo Yu2*, Qing Li3 1Software College, Northeastern University, Shenyang, China 2Institute of Intelligent Manufacturing, Nan Jing Tech University, Nanjing, China 3Peng Cheng Laboratory, Shenzhen, China |
| Pseudocode | Yes | The corresponding pseudocode of BLAQ is presented in Algorithm 1 of the Appendix. ... the corresponding pseudocode is presented in Algorithm 2 of the Appendix. |
| Open Source Code | Yes | You can refer to our appendix on the following website: https://github.com/paper Proof24/Appendix BLAQ |
| Open Datasets | Yes | Four widely-used datasets (CIFAR10, MNIST, SVHN and ILSVRC12 (Image Net)) are applied to validate the performance of BLAQ. |
| Dataset Splits | No | The paper mentions using CIFAR10, MNIST, SVHN, and ILSVRC12 (Image Net) datasets but does not explicitly provide details on train/validation/test splits, specific percentages, or how data was partitioned for reproduction. |
| Hardware Specification | No | The paper discusses experimental results on various datasets and models but does not provide any specific details regarding the hardware specifications (e.g., GPU/CPU models, memory, or computing infrastructure) used for running these experiments. |
| Software Dependencies | No | The paper does not provide specific software dependency names with version numbers (e.g., Python, PyTorch, TensorFlow versions) that would be needed to replicate the experiment environment. |
| Experiment Setup | Yes | Therefore, in BLAQ, we set the hyperparameters a and m to 0.6 and 5 in the datasets CIFAR10, MNIST, and SVHN, and set the hyperparameters a and m to 0.9 and 10 in the dataset ILSVRC12. |