Dual Propagation: Accelerating Contrastive Hebbian Learning with Dyadic Neurons
Authors: Rasmus Høier, D. Staudt, Christopher Zach
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimentally we show on common computer vision datasets, including Imagenet32x32, that dual propagation performs equivalently to back-propagation both in terms of accuracy and runtime. |
| Researcher Affiliation | Academia | 1Chalmers University of Technology, Sweden. Correspondence to: Rasmus Høier <hier@chalmers.se>. |
| Pseudocode | Yes | Algorithm 1 Dual propagation |
| Open Source Code | Yes | Our code is available on github2. 2https://github.com/Rasmuskh/Dual-Propagation |
| Open Datasets | Yes | We evaluate dual propagation on MNIST, CIFAR10, CIFAR100, and Imagenet32x32. [...] The models were trained using standard data augmentation techniques (random-crops and horizontal flips) and the training data of 50 000 images was split into 45 000 for training and 5 000 for validation. [...] As Imagenet32x32 does not have a public test dataset we used the validation data as test data and reserved 5% of the training data for validation. |
| Dataset Splits | Yes | 10% of the training data was reserved for validation, and performance on the validation data was used to select which checkpoint of the model to evaluate on the test dataset. [...] the training data of 50 000 images was split into 45 000 for training and 5 000 for validation. [...] reserved 5% of the training data for validation. |
| Hardware Specification | Yes | On an NVIDIA A100 GPU both BP and DP had runtimes of 3.5 seconds per epoch and 4.5 seconds per epoch for CIFAR10 and CIFAR100 respectively [...] The experiments were enabled by the supercomputing resource Berzelius provided by National Supercomputer Centre at Link oping University and the Knut and Alice Wallenberg foundation. |
| Software Dependencies | No | The paper mentions 'autodiff framework' but does not specify software names with version numbers needed for replication. |
| Experiment Setup | Yes | The hyper-parameters used in the MLP experiments of Section 5.1 are listed in Table 5. [...] The hyper-parameters used in the experiments of Section 5.2 are listed in Table 6. |