Spike-inspired rank coding for fast and accurate recurrent neural networks
Authors: Alan Jeffares, Qinghai Guo, Pontus Stenetorp, Timoleon Moraitis
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate these in two toy problems of sequence classification, and in a temporally-encoded MNIST dataset where our RC model achieves 99.19% accuracy after the first input time-step, outperforming the state of the art in temporal coding with SNNs, as well as in spoken-word classification of Google Speech Commands, outperforming non-RC-trained early inference with LSTMs. |
| Researcher Affiliation | Collaboration | Alan Jeffares1 University College London, UK alan.jeffares.20@ucl.ac.uk Qinghai Guo ACS Lab, Huawei Technologies, Shenzhen, China guoqinghai@huawei.com Pontus Stenetorp University College London, UK pontus@stenetorp.se Timoleon Moraitis Huawei Technologies Zurich, Switzerland timoleon.moraitis@huawei.com |
| Pseudocode | Yes | Algorithm 1 RC-training Given: a training set of N example sequences Si = {xi0, ..., xi T } and corresponding labels yi; an RNN R; and a threshold θ. |
| Open Source Code | No | The paper mentions a 'code implementation' from a cited work (Wu et al., 2018) in Appendix E for SNNs, but it does not state that the authors are releasing their own code for the methodology described in the paper, nor does it provide a direct link to such code. |
| Open Datasets | Yes | The MNIST dataset is a common image-recognition benchmark. ... we used the Google Speech Commands dataset v0.02 (Warden, 2018). |
| Dataset Splits | Yes | An independent validation set of 2000 sequences is evaluated after every 50 training batches. ... retain the model that achieves the highest spiking accuracy on a validation set of size 2000 evaluated every 50 batches. ... The training, validation and testing split of the dataset followed the convention provided in section 7 of Warden (2018). |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running its experiments, such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | The paper mentions the 'Adam optimiser' and uses 'PyTorch' (implied by typical deep learning frameworks), but it does not provide specific version numbers for these or any other software dependencies required to reproduce the experiments. |
| Experiment Setup | Yes | We apply binary cross-entropy loss and Adam optimiser Kingma & Ba (2014) with a learning rate of 0.0003. ... For all results, we used Adam with a learning rate fixed at 0.001, and the threshold of 0.95. The only hyperparameter value that we searched systematically was β. ... We used a network with 128 LSTM units before two fully connected layers of 32 and 11 neurons each, with a softmax at the output. ... We also applied L2-norm gradient-clipping with a maximum of 0.25. |