BMRS: Bayesian Model Reduction for Structured Pruning
Authors: Dustin Wright, Christian Igel, Raghavendra Selvan
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on multiple datasets and neural networks of varying complexity showed that the two BMRS methods offer a competitive performance-efficiency trade-off compared to other pruning methods. |
| Researcher Affiliation | Academia | Dustin Wright, Christian Igel, and Raghavendra Selvan Department of Computer Science, University of Copenhagen {dw,igel,raghav}@di.ku.dk |
| Pseudocode | Yes | Algorithm 1: Training and pruning with BMRS |
| Open Source Code | Yes | Source code: https://github.com/saintslab/bmrs-structured-pruning/ |
| Open Datasets | Yes | We use the following datasets (full details in Appendix B): MNIST [22], Fashion-MNIST [37], CIFAR10 [21], and Tiny Imagenet. |
| Dataset Splits | Yes | We use the original 10,000 image test set for testing and split the 60,000 image train set into 80% training and 20% validation images. |
| Hardware Specification | Yes | All experiments were run on a shared cluster. Requested jobs consisted of 16GB of RAM and 4 Intel Xeon Silver 4110 CPUs. We used a single NVIDIA Titan X GPU with 24GB of RAM for all experiments |
| Software Dependencies | No | The paper mentions the use of the Adam optimizer and implicit use of PyTorch for neural network training but does not provide specific version numbers for these or other software libraries. |
| Experiment Setup | Yes | For each model and dataset we use the Adam optimizer with no weight decay. We train for 50 epochs for each experiment with an MLP and Lenet5, and for 100 epochs for each experiment with Resnet50 and Vi T. Further details about each model are given as follows: MNIST: Number of layers: 7; Hidden dimension: 100; Batch size: 128; Learning rate: 8.5 10 4. |