Towards Energy Efficient Spiking Neural Networks: An Unstructured Pruning Framework
Authors: Xinyu Shi, Jianhao Ding, Zecheng Hao, Zhaofei Yu
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results demonstrate that our method achieves impressive energy efficiency gains. The sparse network pruned by our method with only 0.63% remaining connections can achieve a remarkable 91 times increase in energy efficiency compared to the original dense network, requiring only 8.5M SOPs for inference, with merely 2.19% accuracy loss on the CIFAR-10 dataset. |
| Researcher Affiliation | Academia | Xinyu Shi1,2, Jianhao Ding2, Zecheng Hao2 & Zhaofei Yu1,2 1 Institute for Artificial Intelligence, Peking University. 2 School of Computer Science, Peking University {xyshi,djh01998,haozecheng,yuzf12}@pku.edu.cn |
| Pseudocode | Yes | Algorithm 1: Learning Process |
| Open Source Code | Yes | Codes are available at https://github.com/xyshi2000/Unstructured-Pruning. |
| Open Datasets | Yes | In this section, we evaluate the effectiveness of our method for classification tasks on the CIFAR10 (Krizhevsky et al., 2009), DVS-CIFAR10 (Li et al., 2017), and Image Net (Deng et al., 2009) datasets. CIFAR-10 The CIFAR-10 dataset (Krizhevsky et al., 2009) is a static image dataset. It contains 60,000 samples divided into 10 categories, with 6,000 samples per category. Each category consists of 5,000 training samples and 1,000 testing samples. |
| Dataset Splits | No | The paper specifies training and testing splits for CIFAR-10 and DVS-CIFAR10, but it does not explicitly mention a separate validation split or the methodology for creating one. For CIFAR-10, it mentions "5,000 training samples and 1,000 testing samples." For DVS-CIFAR10, it states "the first 900 samples of each category for training and the last 100 for testing." There is no explicit mention of a validation set split for any dataset. |
| Hardware Specification | No | The paper does not explicitly state the specific hardware (e.g., GPU model, CPU type) used to run the experiments. It mentions various neuromorphic chips like Loihi, True North, and Tian Jic in a discussion about energy models, but these are cited examples of hardware, not the specific hardware used for their experimental training or inference. |
| Software Dependencies | No | The paper mentions optimizers like Adam and SGD but does not provide specific version numbers for these or other software libraries (e.g., Python, PyTorch, TensorFlow, CUDA). |
| Experiment Setup | Yes | For the CIFAR-10 dataset, we use a convolutional SNN structure with 6 convolutional layers and 2 fully-connected layers ([[256C3] 3-MP2] 2-FC 2), similar to the network proposed in previous work (Fang et al., 2021b). Pruning is applied to all neurons and weights in the network, except for neurons in the fully-connected layers. The batch size and simulation step are set to 16 and 8, respectively. We use the Adam optimizer for optimizing both weights w and masks αw, αn. We train the model for a total of 1000 epochs with an initial learning rate of η = 0.0001. The subnetwork is produced at epoch 800, and is then fine-tuned for extra 200 epochs. |