Distribution Adaptive INT8 Quantization for Training CNNs
Authors: Kang Zhao, Sida Huang, Pan Pan, Yinghan Li, Yingya Zhang, Zhenyu Gu, Yinghui Xu3483-3491
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on broad range of computer vision tasks, such as image classification, object detection and video classification, demonstrate that the proposed Distribution Adaptive INT8 Quantization training method has achieved almost lossless training accuracy for different backbones, including Res Net, Mobile Net V2, Inception V3, VGG and Alex Net, which is superior to the state-of-the-art techniques. |
| Researcher Affiliation | Industry | Kang Zhao, Sida Huang, Pan Pan, Yinghan Li, Yingya Zhang, Zhenyu Gu, Yinghui Xu Machine Intelligence Technology Lab, Alibaba Group {zhaokang.zk, sida.hsd, panpan.pp, lyh238099, yingya.zyy, zhenyu.gu, renji.xyh}@alibaba-inc.com |
| Pseudocode | Yes | Algorithm 1 Backward pass of Distribution Adaptive INT8 Quantization. |
| Open Source Code | No | The paper does not provide an explicit statement or link indicating that their source code is publicly available. |
| Open Datasets | Yes | We conduct experiments on CIFAR-10 (Krizhevsky et al. 2009) and Image Net (Deng et al. 2009) datasets... Faster-RCNN (Ren et al. 2015) and Retina Net (Lin et al. 2017a) on COCO (Lin et al. 2014) and PASCAL VOC datasets (Everingham et al. 2010)... UCF-101 (Soomro, Zamir, and Shah 2012) with official split-1 and Kinetics-400 (Kay et al. 2017). |
| Dataset Splits | No | While the paper uses standard public datasets, it does not explicitly provide specific percentages, sample counts, or detailed methodology for training/validation/test dataset splits needed for reproduction beyond mentioning 'official split-1' for UCF-101 without detailing the validation part of that split. |
| Hardware Specification | Yes | Running time of Res Net-50 on Ge Force RTX 2080Ti (batch size = 64) |
| Software Dependencies | Yes | Our INT8 implementation over FP16 version (based on cu DNN 7.6.5) |
| Experiment Setup | Yes | We use cosine scheduler (Loshchilov and Hutter 2017) for Mobile Net V2 (Sandler et al. 2018), and multistep learning rate scheduler for the others. All experimental settings for INT8 training are consistent with the full precision model (baseline)... In the following experiments, we set k = 1 and A = 0.8... We adopt the open-source MMdetection framework (Chen et al. 2019) with the default settings and take Res Net-50 as backbone. |