A Unified Framework for Soft Threshold Pruning
Authors: Yanqi Chen, Zhengyu Ma, Wei Fang, Xiawu Zheng, Zhaofei Yu, Yonghong Tian
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments and verify its state-of-the-art performance on both Artificial Neural Networks (Res Net-50 and Mobile Net-V1) and Spiking Neural Networks (SEW Res Net-18) on Image Net datasets. |
| Researcher Affiliation | Academia | Yanqi Chen1,3, Zhengyu Ma 3, Wei Fang1,3, Xiawu Zheng3, Zhaofei Yu1,2,3, Yonghong Tian 1,3 1National Engineering Research Center of Visual Technology, School of Computer Science, Peking University; 2Institute for Artificial Intelligence, Peking University; 3Peng Cheng Laboratory yhtian@pku.edu.cn, mazhy@pcl.ac.cn; |
| Pseudocode | Yes | Algorithm 1 The general form of soft threshold pruning algorithm coupled with vanilla SGD (STR and STDS for instance). |
| Open Source Code | Yes | The code is available at https://github.com/Yanqi-Chen/LATS. |
| Open Datasets | Yes | We conduct extensive experiments and verify its state-of-the-art performance on both Artificial Neural Networks (Res Net-50 and Mobile Net-V1) and Spiking Neural Networks (SEW Res Net-18) on Image Net datasets. (Deng et al., 2009) |
| Dataset Splits | No | The paper states it uses the ImageNet dataset and describes training hyperparameters like batch size and epochs but does not explicitly provide the train/validation/test dataset splits (e.g., percentages or sample counts) used for reproduction. |
| Hardware Specification | No | The paper mentions 'The computing resources of Pengcheng Cloudbrain are used in this research' in the acknowledgments but does not provide specific details about the hardware (e.g., GPU models, CPU types, or memory specifications) used for running the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., Python 3.x, PyTorch 1.x) that would be needed for reproduction. |
| Experiment Setup | Yes | In all experiments we switch sparisty levels by changing D, which is equivalent to tuning L1 penalty coefficient. We also tune hyperparameter β in PGH scheduler to maintain different phases of early pruning algorithm. Table 7: ANN Res Net-50 hyperparameters. Table 8: ANN Mobile Net-V1 hyperparameters. Table 9: SNN SEW Res Net-18 hyperparameters. Each table lists values for # Epoch, Optimizer, Overall batch size, Max. learning rate, Learning rate scheduler, Warmup epochs, Label smoothing, Weight decay, and other relevant settings. |