Position-based Scaled Gradient for Model Quantization and Pruning
Authors: Jangho Kim, KiYoon Yoo, Nojun Kwak
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results on CIFAR10/100 and Image Net datasets show the effectiveness of the proposed PSG in both domains of pruning and quantization even for extremely low bits. |
| Researcher Affiliation | Academia | Jangho Kim Seoul National University Seoul, Korea kjh91@snu.ac.kr Ki Yoon Yoo Seoul National University Seoul, Korea 961230@snu.ac.kr Nojun Kwak Seoul National University Seoul, Korea nojunk@snu.ac.kr |
| Pseudocode | No | The paper describes its proposed method mathematically and in prose but does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is released in Github2. 2https://github.com/Jangho-Kim/PSG-pytorch |
| Open Datasets | Yes | The experimental results on CIFAR10/100 and Image Net datasets show the effectiveness of the proposed PSG in both domains of pruning and quantization even for extremely low bits. |
| Dataset Splits | No | The paper does not explicitly provide the training/validation/test dataset splits, only stating that it followed settings from other papers like [1] for CIFAR-10 and ImageNet. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | We used the Pytorch framework for all experiments. |
| Experiment Setup | Yes | PSGD is applied from the last 15 epochs for Image Net experiments and from the first learning rate decay epoch for CIFAR experiments. We use additional 30 epochs for PSGD at extremely low bits experiments (Table 4). Also, we tuned the hyper-parameter λs for each bit-widths and sparsity. |