Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples
Authors: Kanghyun Choi, Deokki Hong, Noseong Park, Youngsok Kim, Jinho Lee
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results show that Qimera achieves state-of-the-art performances for various settings on data-free quantization. |
| Researcher Affiliation | Academia | Kanghyun Choi1 Deokki Hong2 Noseong Park1,2 Youngsok Kim1,2 Jinho Lee1,2 1 Department of Computer Science, Yonsei University 2 Department of Artificial Intelligence, Yonsei University {kanghyun.choi, dk.hong, noseong, youngsok, leejinho}@yonsei.ac.kr |
| Pseudocode | No | No pseudocode or algorithm blocks are present in the paper. |
| Open Source Code | Yes | Code is available at https://github.com/iamkanghyunchoi/qimera. |
| Open Datasets | Yes | Our method is evaluated on CIFAR-10, CIFAR-100 [35] and Image Net (ILSVRC2012 [41]) datasets, which are well-known datasets for evaluating the performance of a model on the image classification task. |
| Dataset Splits | Yes | CIFAR-10/100 datasets consist of 50k training sets and 10k evaluation sets with 10 classes and 100 classes, respectively, and is used for small-scale experiments in our evaluation. Image Net dataset is used for large-scale experiments. It has 1.2 million training sets and 50k evaluation sets. To keep the data-free environment, only the evaluation sets were used for test purposes in all experiments. |
| Hardware Specification | Yes | We implemented all our techniques using Py Torch [43] and ran the experiments using RTX3090 GPUs. |
| Software Dependencies | No | The paper mentions 'Py Torch [43]' and 'pytorchcv library [44]' but does not provide specific version numbers for these or any other ancillary software components, which are necessary for full reproducibility. |
| Experiment Setup | Yes | For CIFAR, the intermediate embedding dimension after the disentanglement mapping layer was set as 64. The generator was trained using Adam [45] with a learning rate of 0.001. For Image Net, the intermediate embedding dimension was set to be 100. ... To fine-tune the quantized model Q, we used SGD with Nesterov [48] as an optimizer for Q with a learning rate of 0.0001 while momentum and weight decay terms as 0.9 and 0.0001 respectively. The generator G and the quantized model Q were jointly trained with 200 iterations for 400 epochs while decaying the learning rate by 0.1 per every 100 epochs. The batch size was 64 and 16 for CIFAR and Image Net respectively. ...loss functions L(G) and L(Q) are equal to Eq. 1 and Eq. 3 with α = 0.1 and δ = 1.0, following the baseline. |