Deep Learning without Weight Transport
Authors: Mohamed Akrout, Collin Wilson, Peter Humphreys, Timothy Lillicrap, Douglas B. Tweed
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Tested on the Image Net visual-recognition task, these mechanisms learn almost as well as backprop (the standard algorithm of deep learning, which uses weight transport) and they outperform feedback alignment and another, more-recent transport-free algorithm, the sign-symmetry method. |
| Researcher Affiliation | Collaboration | Mohamed Akrout University of Toronto, Triage Collin Wilson University of Toronto Peter C. Humphreys Deep Mind Timothy Lillicrap Deep Mind, University College London Douglas Tweed University of Toronto, York University |
| Pseudocode | No | The paper provides equations and diagrams of neural circuits but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | We have released a Python version of the proprietary Tensor Flow/TPU code for the weight mirror and the KP reciprocal network that we used in our tests; see github.com/makrout/Deep-Learning-without-Weight-Transport. |
| Open Datasets | Yes | Tested on the Image Net visual-recognition task [6] |
| Dataset Splits | No | The paper mentions ImageNet and discussions of hyperparameter selection in Appendix D.1, implying a standard validation process. However, it does not explicitly state the specific percentages or sample counts for training, validation, or test splits in the main text needed for reproduction. |
| Hardware Specification | Yes | We have released a Python version of the proprietary Tensor Flow/TPU code for the weight mirror and the KP reciprocal network that we used in our tests; see github.com/makrout/Deep-Learning-without-Weight-Transport. |
| Software Dependencies | No | The paper mentions 'Python version' and 'Tensor Flow/TPU code' but does not specify exact version numbers for these software components. |
| Experiment Setup | No | The paper mentions the use of 'Res Net block variant', 'Batch Norm', 'Re LUs', 'learning rate ηW', and 'weight-decay factor λ', and refers to 'Appendix D.1 for details of our hyperparameter selection'. However, it does not provide concrete hyperparameter values or detailed training configurations directly in the main text. |