A2Q+: Improving Accumulator-Aware Weight Quantization
Authors: Ian Colbert, Alessandro Pappalardo, Jakoba Petri-Koenig, Yaman Umuroglu
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We identify and characterize the various tradeoffs that arise as a consequence of accumulator constraints and support our analysis with experiments that show A2Q+ significantly improves these trade-offs when compared to prior methods. |
| Researcher Affiliation | Industry | 1AMD SW Technology Team, San Diego, California, USA 2AMD Research and Advanced Development, Dublin, Ireland. Correspondence to: Ian Colbert <ian.colbert@amd.com>. |
| Pseudocode | No | The paper describes algorithms and mathematical formulations but does not include a distinct block labeled "Pseudocode" or "Algorithm" with structured steps. |
| Open Source Code | No | The paper states: "We implement A2Q+ in Py Torch (Paszke et al., 2019) using v0.10 of the Brevitas quantization library (Pappalardo, 2021)", which indicates usage of an open-source library, but there is no explicit statement or link confirming that the authors' own implementation code for A2Q+ is open-source or publicly available. |
| Open Datasets | Yes | In Section 4.1, we evaluate Mobile Net V1 (Howard et al., 2017) and Res Net18 (He et al., 2016) trained on the CIFAR-10 dataset (Krizhevsky et al., 2009) for image classification, and ESPCN (Shi et al., 2016) and U-Net (Ronneberger et al., 2015) trained on the BSD300 dataset (Martin et al., 2001) for super resolution. In Section 4.2, we evaluate larger image classification benchmarks, namely Res Net18, Res Net34, and Res Net50 trained on the Image Net-1K dataset (Deng et al., 2009). |
| Dataset Splits | No | The paper mentions using "calibration sets" (e.g., "using a calibration set of 3000 images randomly sampled from the training dataset") for graph equalization and bias correction, which is a preprocessing step, but it does not specify a distinct validation split for monitoring training progress or hyperparameter tuning in the traditional sense of a train/validation/test split. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments, such as GPU models, CPU types, or cloud computing instance specifications. |
| Software Dependencies | Yes | We implement A2Q+ in Py Torch (Paszke et al., 2019) using v0.10 of the Brevitas quantization library (Pappalardo, 2021) and leverage their implementations of A2Q and baseline QAT methods for benchmarking. |
| Experiment Setup | Yes | We include all training details and hyperparameters in Appendix B.1. |