Fixed Point Quantization of Deep Convolutional Networks
Authors: Darryl Lin, Sachin Talathi, Sreekanth Annapureddy
ICML 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that in comparison to equal bit-width settings, the fixed point DCNs with optimized bit width allocation offer > 20% reduction in the model size without any loss in accuracy on CIFAR-10 benchmark. We also demonstrate that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. In doing so, we report a new state-of-the-art fixed point performance of 6.78% error-rate on CIFAR-10 benchmark. |
| Researcher Affiliation | Industry | Darryl D. Lin DARRYL.DLIN@GMAIL.COM Qualcomm Research, San Diego, CA 92121, USA Sachin S. Talathi TALATHI@GMAIL.COM Qualcomm Research, San Diego, CA 92121, USA V. Sreekanth Annapureddy SREEKANTHAV@GMAIL.COM Netra Dyne Inc., San Diego, CA 92121, USA |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | Our experiments show that in comparison to equal bit-width settings, the fixed point DCNs with optimized bit width allocation offer > 20% reduction in the model size without any loss in accuracy on CIFAR-10 benchmark. We also demonstrate that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. In doing so, we report a new state-of-the-art fixed point performance of 6.78% error-rate on CIFAR-10 benchmark. ... Here we carry out a similar exercise for an Alex Netlike DCN that is trained on Image Net-1000. |
| Dataset Splits | No | The paper mentions using CIFAR-10 and ImageNet benchmarks, which typically have predefined splits, but it does not explicitly state the specific training, validation, and test split percentages or sample counts used for reproduction. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU/CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper does not list any specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow, CUDA versions) that would be needed to replicate the experiments. |
| Experiment Setup | Yes | Table 2. Parameters per layer in our CIFAR-10 network ... Table 4. Parameters per layer in our Alex Net implementation ... Table 7 contains the classification error rate (in %) for the CIFAR-10 network after fine-tuning the model for 30 epochs. We experiment with different weight and activation bit-width combinations, ranging from floating point to 4-bit, 8-bit, and 16-bit fixed point. |