TexQ: Zero-shot Network Quantization with Texture Feature Distribution Calibration
Authors: Xinrui Chen, Yizhi Wang, Renao YAN, Yiqing Liu, Tian Guan, Yonghong He
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on CIFAR10/100 and Image Net show that Tex Q is observed to perform state-of-the-art in low bit width quantization. |
| Researcher Affiliation | Academia | Xinrui Chen, Yizhi Wang, Renao Yan , Yiqin Liu, Tian Guan , Yonghong He Tsinghua Shenzhen International Graduate School, Tsinghua University {cxr22, yz-wang22, yra21, liuyiqin20}@mails.tsinghua.edu.cn {guantian, heyh}@sz.tsinghua.edu.cn |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described in this paper. |
| Open Datasets | Yes | We report top-1 accuracy on validation sets of CIFAR-10/100 [52] and Image Net [53]. |
| Dataset Splits | Yes | We report top-1 accuracy on validation sets of CIFAR-10/100 [52] and Image Net [53]. In fine-tuning, batchsize is 128 for CIFAR-10/100 and 16 for Image Net, adjusting by cosine annealing [57]. |
| Hardware Specification | Yes | All experiments are implemented with Pytorch [56] via the code of FDDA [19] and Intra Q [29], and run on an NVIDIA Ge Force RTX 3090 GPU. |
| Software Dependencies | No | The paper mentions 'Pytorch' but does not specify its version number or any other software dependencies with their versions. |
| Experiment Setup | Yes | Calibration images are iterated with a constant learning rate of 0.05. Generator is imported from GDFQ [24] with a initial learning rate of 1e-3 multiplied by 0.1 every 100 epochs. In fine-tuning, batchsize is 128 for CIFAR-10/100 and 16 for Image Net, adjusting by cosine annealing [57]. We warm up G for 50 epochs, then update G and Q for 450 epochs. The optimal configurations on trade-off parameters from α1 to α6 obtained by grid search are 2, 10, 0.4, 0.02, 1.8, and 20. The optimal configurations are k=9, θU=0.3, θL=0.5, and ε=0.015. |