Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization
Authors: Yamato Arai, Yuma Ichikawa
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on several LLMs demonstrate that QEP-enhanced layer-wise PTQ achieves substantially higher accuracy than existing methods. Notably, the gains are most pronounced in the extremely low-bit quantization regime. |
| Researcher Affiliation | Collaboration | Yamato Arai Fujitsu Limited Department of Basic Science The University of Tokyo Yuma Ichikawa Fujitsu Limited RIKEN center for AIP |
| Pseudocode | No | The paper describes methods using mathematical formulations (e.g., Eq. 1, 3, 5, 6) and textual descriptions, but does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code: https://github.com/Fujitsu Research/qep |
| Open Datasets | Yes | Specifically, GPTQ and Qu IP use the C4 dataset [Frantar et al., 2022] for calibration, while AWQ uses the Pile dataset [Gao et al., 2020]. Following Frantar et al. [2022], we evaluate the correction term in Eq. (4) using 128 randomly sampled segments of 2048 tokens each from the C4 dataset[Raffel et al., 2020]... We evaluate the quantized LLMs using the perplexity (PPL) on Wiki Text2 [Merity et al., 2016], Penn Treebank (PTB) [Marcus et al., 1994], and C4 [Raffel et al., 2020], and zero-shot accuracy on benchmarks including ARC Easy (Arc E) [Boratko et al., 2018], Pi QA [Bisk et al., 2020], and Story Cloze (SC) [Mostafazadeh et al., 2016]. |
| Dataset Splits | Yes | Following established evaluation protocols from prior studies [Dettmers et al., 2022, Xiao et al., 2023, Frantar et al., 2022, Dettmers and Zettlemoyer, 2023, Yao et al., 2022], we evaluate the quantized LLMs using the perplexity (PPL) on Wiki Text2 [Merity et al., 2016], Penn Treebank (PTB) [Marcus et al., 1994], and C4 [Raffel et al., 2020], and zero-shot accuracy on benchmarks including ARC Easy (Arc E) [Boratko et al., 2018], Pi QA [Bisk et al., 2020], and Story Cloze (SC) [Mostafazadeh et al., 2016]. |
| Hardware Specification | Yes | All experiments are conducted using a single NVIDIA V100 GPU. |
| Software Dependencies | No | The main paper text does not explicitly list specific software dependencies with version numbers. |
| Experiment Setup | Yes | For the propagation strength parameter αl, we adopt a representative default value of αl = 1/2 for all layers, except for the MLP layers in the Llama-2 70B model, for which we set αl = 0. Tuning αl can further improve performance but is beyond the scope of this study and is left for future work. |