Data-Independent Neural Pruning via Coresets
Authors: Ben Mussay, Margarita Osadchy, Vladimir Braverman, Samson Zhou, Dan Feldman
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the effectiveness of our method on popular network architectures. In particular, our coresets yield 90% compression of the Le Net-300-100 architecture on MNIST while improving classification accuracy. and 4 EXPERIMENTS |
| Researcher Affiliation | Academia | Ben Mussay Computer Science Department University of Haifa Haifa, Israel bengordoncshaifa@gmail.com Margarita Osadchy Computer Science Department University of Haifa Haifa, Israel rita@cs.haifa.ac.il Vladimir Braverman Computer Science Department Johns Hopkins University Baltimore, MD., USA vova@cs.jhu.edu Samson Zhou Computer Science Department Carnegie Mellon University Pittsburgh, IN., USA samsonzhou@gmail.com Dan Feldman Computer Science Department University of Haifa Haifa, Israel dannyf.post@gmail.com |
| Pseudocode | Yes | Algorithm 1: CORESET(P, w, m, φ, β) and Algorithm 2: CORESET PER LAYER(P, w1, , wk, m, φ, β) |
| Open Source Code | Yes | The source code of our method can be found at: https://github.com/Ben Mussay/Data-Independent Neural-Pruning-via-Coresets. |
| Open Datasets | Yes | Le Net-300-100 on MNIST (Le Cun et al., 1998), and VGG-16 (Simonyan & Zisserman, 2014) on CIFAR-10 (Krizhevsky, 2009). |
| Dataset Splits | No | The paper mentions using 'MNIST test set as queries' and training on MNIST and CIFAR-10, but it does not specify explicit train/validation/test dataset splits for its own experiments, nor does it explicitly mention a validation set being used for its models during training or evaluation. |
| Hardware Specification | Yes | The experiments were implemented in Py Torch (Paszke et al., 2017) on a Linux Machine using an Intel Xeon, 32-core CPU with 3.2 GHz, 256 GB of RAM and Nvidia Titan X and Quadro M4000 GPUs. |
| Software Dependencies | No | The paper mentions 'Py Torch (Paszke et al., 2017)' but does not specify a version number for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | For every layer, after applying neural pruning the remaining weights were fine-tuned until convergence. and We ran three tests, varying the distribution of weights. In the first and second tests (Figure 2, (a) and (b)) the weights were drawn from the Gaussian and Uniform distributions respectively. The total number of neurons was set to 1000. We selected subsets of neurons of increasing sizes from 50 to 1000 with a step of 50. In the third test (Figure 2, (c)) we used the trained weights from the first layer of Lenet-300-100 including 300 neurons. We varied the coreset size from 50 to 300 with a step 50. |