Group Fisher Pruning for Practical Network Compression
Authors: Liyang Liu, Shilong Zhang, Zhanghui Kuang, Aojun Zhou, Jing-Hao Xue, Xinjiang Wang, Yimin Chen, Wenming Yang, Qingmin Liao, Wayne Zhang
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on various backbones, including the classic Res Net and Res Ne Xt, mobilefriendly Mobile Net V2, and the NAS-based Reg Net, both on image classification and object detection which is under-explored. Experimental results validate that our method can effectively prune sophisticated networks, boosting inference speed without sacrificing accuracy. |
| Researcher Affiliation | Collaboration | 1Shenzhen International Graduate School/Department of Electronic Engineering, Tsinghua University, Beijing, China 2Shanghai AI Laboratory, Shanghai, China 3Sense Time Research, Hong Kong, China 4Department of Statistical Science, University College London, London, United Kingdom 5Qing Yuan Research Institute, Shanghai, China. |
| Pseudocode | Yes | Algorithm 1 Layer Grouping; Algorithm 2 Group Fisher Pruning |
| Open Source Code | Yes | The code will be available at https://github.com/jshilong/FisherPruning. |
| Open Datasets | Yes | We conduct all experiments for the task of image classification and object detection on the Image Net (Deng et al., 2009) and COCO (Lin et al., 2014) datasets, respectively. |
| Dataset Splits | Yes | We conduct all experiments for the task of image classification and object detection on the Image Net (Deng et al., 2009) and COCO (Lin et al., 2014) datasets, respectively. ... The column T1 represents top-1 accuracy on the validation set |
| Hardware Specification | Yes | We measure the batch inference time on NVIDIA 2080 Ti GPU |
| Software Dependencies | No | All experiments are done using Py Torch (Paszke et al., 2019). The paper mentions PyTorch but does not specify a version number for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | We prune a channel every d = 25/10 iterations when pruning classification/detection networks. After the whole pruning process we fine-tune the pruned model for the same number of epochs that is used to train the unpruned model, which is trained following standard practices. |