SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY
Authors: Namhoon Lee, Thalaiyasingam Ajanthan, Philip Torr
ICLR 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method on MNIST, CIFAR-10, and Tiny-Image Net classification tasks with a variety of network architectures. Our results show that SNIP yields extremely sparse models with minimal or no loss in accuracy across all tested architectures, while being much simpler than other state-of-the-art alternatives. |
| Researcher Affiliation | Academia | Namhoon Lee, Thalaiyasingam Ajanthan & Philip H. S. Torr University of Oxford {namhoon,ajanthan,phst}@robots.ox.ac.uk |
| Pseudocode | Yes | Algorithm 1 SNIP: Single-shot Network Pruning based on Connection Sensitivity |
| Open Source Code | Yes | The code can be found here: https://github.com/namhoonlee/snip-public. |
| Open Datasets | Yes | We evaluate our method on MNIST, CIFAR-10, and Tiny-Image Net classification tasks with a variety of network architectures. |
| Dataset Splits | Yes | We spare 10% of the training data as a validation set and used only 90% for training. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions optimizers like SGD and Adam but does not provide specific version numbers for any software dependencies or deep learning frameworks. |
| Experiment Setup | Yes | Specifically, we train the models using SGD with momentum of 0.9, batch size of 100 for MNIST and 128 for CIFAR experiments and the weight decay rate of 0.0005, unless stated otherwise. The initial learning rate is set to 0.1 and decayed by 0.1 at every 25k or 30k iterations for MNIST and CIFAR, respectively. |