Spike-driven Transformer
Authors: Man Yao, JiaKui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, Guoqi Li
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments show that the proposed architecture can outperform or comparable to State Of-The-Art (SOTA) SNNs on both static and neuromorphic datasets. We achieved 77.1% accuracy on Image Net-1K, which is the SOTA result in the SNN field. |
| Researcher Affiliation | Academia | Man Yao1,2 , Jiakui Hu3,1 , Zhaokun Zhou3,2 , Li Yuan3,2, Yonghong Tian3,2, Bo Xu1, Guoqi Li1 1Institute of Automation, Chinese Academy of Sciences, Beijing, China 2Peng Cheng Laboratory, Shenzhen, Guangzhou, China 3Peking University, Beijing, China |
| Pseudocode | No | No structured pseudocode or algorithm blocks found. |
| Open Source Code | Yes | All source code and models are available at https://github.com/BICLab/Spike-Driven-Transformer. |
| Open Datasets | Yes | We evaluate our method on both static datasets Image Net [77], CIFAR-10/100 [78], and neuromorphic datasets CIFAR10-DVS [79], DVS128 Gesture [80]. |
| Dataset Splits | No | The paper mentions "Standard data augmentation techniques, like random augmentation, mixup, are also employed in training. Details of the training and experimental setup on Image Net are given in the supplementary material." and "We basically keep the experimental setup in [20], including the network structure, training settings, etc., and details are given in the supplementary material." It does not provide explicit train/validation splits in the main body of the paper. |
| Hardware Specification | No | No specific hardware details (like GPU model, CPU model, or memory) used for running experiments are provided in the paper. |
| Software Dependencies | No | The paper mentions "The optimizer is Lamb." but does not provide specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | The input size is set to 224 224. The batch size is set to 128 or 256 during 310 training epochs with a cosine-decay learning rate whose initial value is 0.0005. The optimizer is Lamb. The image is divided into N = 196 patches using the SPS module. Standard data augmentation techniques, like random augmentation, mixup, are also employed in training. |