Hebbian Deep Learning Without Feedback
Authors: Adrien Journé, Hector Garcia Rodriguez, Qinghai Guo, Timoleon Moraitis
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | accuracies on MNIST, CIFAR-10, STL-10, and Image Net, respectively reach 99.4%, 80.3%, 76.2%, and 27.3%. In conclusion, Soft Hebb shows with a radically different approach from BP that Deep Learning over few layers may be plausible in the brain and increases the accuracy of bio-plausible machine learning. Code is available at https://github.com/Neuromorphic Computing/Soft Hebb. |
| Researcher Affiliation | Industry | Adrien Journ e1, Hector Garcia Rodriguez1, Qinghai Guo2, Timoleon Moraitis1* {adrien.journe, hector.garcia.rodriguez, guoqinghai, timoleon.moraitis}@huawei.com 1Huawei Zurich Research Center, Switzerland 2Huawei ACS Lab, Shenzhen, China |
| Pseudocode | No | The paper describes the Soft Hebb algorithm and its plasticity rule using equations, but it does not include pseudocode or a clearly labeled algorithm block. |
| Open Source Code | Yes | Code is available at https://github.com/Neuromorphic Computing/Soft Hebb. |
| Open Datasets | Yes | accuracies on MNIST, CIFAR-10, STL-10, and Image Net, respectively reach 99.4%, 80.3%, 76.2%, and 27.3%. |
| Dataset Splits | Yes | All grid searches were performed on three different random seeds, varying the batch sampling and the validation set (20% of the training dataset). |
| Hardware Specification | Yes | We used an NVIDIA Tesla V100 32GB GPU. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies used in the experiments. It only mentions "PyTorch" in the context of another paper's code (Appendix A.7). |
| Experiment Setup | Yes | The linear classifier on top uses a mini-batch of 64 and trains on 50 epochs for MNIST and CIFAR-10, 100 epochs for STL-10, and 200 epochs for Image Net. For all datasets, the learning-rate has an initial value of 0.001 and is halved repeatedly at [20%, 35%, 50%, 60%, 70%, 80%, 90%] of the total number of epochs. Data augmentation (random cropping and flipping) was applied for STL-10 and Image Net. |