Towards Higher Ranks via Adversarial Weight Pruning
Authors: Yuchuan Tian, Hanting Chen, Tianyu Guo, Chao Xu, Yunhe Wang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on various datasets and different tasks demonstrate the effectiveness of our algorithm in high sparsity. |
| Researcher Affiliation | Collaboration | Yuchuan Tian1, Hanting Chen2, Tianyu Guo2, Chao Xu1, Yunhe Wang2 1 National Key Lab of General AI, School of Intelligence Science and Technology, Peking University. 2 Huawei Noah s Ark Lab. |
| Pseudocode | Yes | The whole pruning framework is detailed in Algorithm 1. |
| Open Source Code | Yes | The codes are available at https://github.com/huawei-noah/Efficient-Computing/tree/ master/Pruning/RPG and https://gitee.com/mindspore/models/tree/ master/research/cv/RPG. |
| Open Datasets | Yes | CIFAR-10 is one of the most widely used benchmark for image classification. It consists of 60000 32 32 images: 50000 for training, and 10000 for validation. Image Net ISLVRC2012 [9] is a large scale image classification dataset. Mask R-CNN pruning on COCO val2017. |
| Dataset Splits | Yes | CIFAR-10... 50000 for training, and 10000 for validation. Image Net ISLVRC2012 [9]... It contains 1281K images in the training set and 50K images in the validation set. |
| Hardware Specification | Yes | Image Net experiments are run on 8 NVIDIA Tesla V100s. |
| Software Dependencies | No | We gratefully acknowledge the support of Mind Spore [38], CANN (Compute Architecture for Neural Networks) and Ascend AI Processor used for this research. (No version numbers provided for MindSpore or CANN). |
| Experiment Setup | Yes | We use the SGD optimizer with momentum 0.9, batchsize 128, learning rate 0.1, and weight decay 0.005. |