Redistribution of Weights and Activations for AdderNet Quantization
Authors: Ying Nie, Kai Han, Haikang Diao, Chuanjian Liu, Enhua Wu, Yunhe Wang
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The effectiveness of the proposed quantization method for Adder Net is well verified on several benchmarks, e.g. , our 4-bit post-training quantized adder Res Net-18 achieves an 66.5% top-1 accuracy on the Image Net with comparable energy efficiency, which is about 8.5% higher than that of the previous Adder Net quantization methods. |
| Researcher Affiliation | Collaboration | 1Huawei Noah s Ark Lab 2State Key Lab of Computer Science, ISCAS & UCAS 3School of Integrated Circuits, Peking University 4University of Macau {ying.nie, kai.han, yunhe.wang}@huawei.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code will be available at https://gitee.com/ mindspore/models/tree/master/research/cv/Adder Quant. |
| Open Datasets | Yes | The CIFAR-10/100 dataset consists of 60,000 color images in 10/100 classes with 32 32 size, including 50,000 training and 10,000 validation images. We further conduct evaluation on Image Net dataset [9], which contains over 1.2M training images and 50K validation images belonging to 1,000 classes. |
| Dataset Splits | Yes | The CIFAR-10/100 dataset consists of 60,000 color images in 10/100 classes with 32 32 size, including 50,000 training and 10,000 validation images. We further conduct evaluation on Image Net dataset [9], which contains over 1.2M training images and 50K validation images belonging to 1,000 classes. |
| Hardware Specification | Yes | We gratefully acknowledge the support of Mind Spore [17], CANN(Compute Architecture for Neural Networks) and Ascend AI Processor used for this research. |
| Software Dependencies | No | The paper mentions the use of 'Mind Spore' but does not provide specific version numbers for MindSpore or any other key software dependencies. |
| Experiment Setup | Yes | We employ learning rate starting at 0.1 and decay the learning rate with cosine learning rate schedule. SGD with momentum of 0.9 is adopted as our optimization algorithm. The weight decay is set to 5 10 4. We train the models for 400 epochs with a batch size of 256. |