TAB: Temporal Accumulated Batch Normalization in Spiking Neural Networks
Authors: Haiyan Jiang, Vincent Zoonekynd, Giulia De Masi, Bin Gu, Huan Xiong
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on CIFAR-10, CIFAR-100, and DVS-CIFAR10 show that our TAB method outperforms other state-of-the-art methods. |
| Researcher Affiliation | Collaboration | 1Mohamed bin Zayed University of Artificial Intelligence, UAE 2Abu Dhabi Investment Authority, UAE 3Technology Innovation Institute, UAE 4Sant Anna School of Advanced Studies, Italy 5Harbin Institute of Technology, China 6School of Artificial Intelligence, Jilin University, China |
| Pseudocode | Yes | For the algorithm details of the TAB method, please refer to algorithm 1 in the Appendix. ... Algorithm 1: The Algorithm of TAB Method |
| Open Source Code | Yes | Corresponding authors. Codes are available at https://github.com/HaiyanJiang/SNN-TAB. |
| Open Datasets | Yes | We conduct extensive experiments on large-scale static and neuromorphic datasets, CIFAR-10 (Krizhevsky et al., 2009), CIFAR100 (Krizhevsky et al., 2009), and DVS-CIFAR10 (Li et al., 2017), to verify the effectiveness of our proposed TAB method. |
| Dataset Splits | Yes | The CIFAR-10 dataset (Krizhevsky et al., 2009) consists of 60, 000 color images each with image size of 32 32 in 10 classes of objects such as airplanes, cars, and birds, with 6, 000 images per class. There are 50, 000 samples in the training set and 10, 000 samples in the test set. ... The CIFAR-100 dataset (Krizhevsky et al., 2009) consists of 60, 000 32 32 color images in 100 classes with 6, 000 images per class. There are 50, 000 samples in the training set and 10, 000 samples in the test set. ... Following Samadzadeh et al. (2023); Duan et al. (2022), we split the dataset into 9, 000 training images and 1, 000 testing images, and reduce the spatial resolution from the original 128 128 to 48 48. |
| Hardware Specification | Yes | Training occurs on an NVIDIA RTX A6000 with 4 GPUs, each handling a batch size of 24. ... All of our models are trained on the PyTorch platform. Experiments are conducted on an NVIDIA RTX A6000 GPU. |
| Software Dependencies | No | The paper mentions 'PyTorch platform' but does not specify a version number. No other software dependencies are listed with specific versions. |
| Experiment Setup | Yes | The network is trained using the Adam W optimizer with an initial learning rate of 0.00002 and a weight decay of 0.02. Training occurs on an NVIDIA RTX A6000 with 4 GPUs, each handling a batch size of 24. ... The Stochastic Gradient Descent (SGD) optimizer (Bottou, 2012) is used in the experiments with a momentum parameter of 0.9. We use a cosine decay scheduler (Loshchilov & Hutter, 2017) to adjust the learning rate with a weight decay 5 10 4 for CIFAR-10/CIFAR-100 datasets. All models are trained for 200 epochs. We set the initial learning rate to ϵ = 0.01 for CIFAR-10 and CIFAR-100. ... Other hyperparameters can be found in Table S2. |