Towards the Limit of Network Quantization
Authors: Yoojin Choi, Mostafa El-Khamy, Jungwon Lee
ICLR 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, using the simple uniform quantization followed by Huffman coding, we show from our experiments that the compression ratios of 51.25, 22.17 and 40.65 are achievable for Le Net, 32-layer Res Net and Alex Net, respectively. |
| Researcher Affiliation | Industry | Yoojin Choi, Mostafa El-Khamy, and Jungwon Lee Samsung US R&D Center, San Diego, CA 92121, USA {yoojin.c,mostafa.e,jungwon2.lee}@samsung.com |
| Pseudocode | Yes | Algorithm 1 Iterative solution for entropy-constrained network quantization |
| Open Source Code | No | The paper does not provide any links to source code repositories, nor does it state that its code is publicly available or in supplementary materials. |
| Open Datasets | Yes | Le Net (Le Cun et al., 1998) for the MNIST data set, (b) Res Net (He et al., 2015) for the CIFAR-10 data set, and (c) Alex Net (Krizhevsky et al., 2012) for the Image Net ILSVRC-2012 data set (Russakovsky et al., 2015). |
| Dataset Splits | Yes | For the data set X used to compute Hessian in (8), we can either reuse a training data set or use some other data set, e.g., validation data set. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU or CPU models, memory, or cloud computing instances used for the experiments. |
| Software Dependencies | No | The paper mentions using a 'pre-trained Alex Net Caffe model' and refers to the 'Adam SGD optimizer (Kingma & Ba, 2014)' but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | For Hessian computation, 50,000 samples of the training set are reused. We also evaluate the performance when Hessian is computed with 1,000 samples only. |