Improving equilibrium propagation without weight symmetry through Jacobian homeostasis
Authors: Axel Laborieux, Friedemann Zenke
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | An empirical demonstration that h EP with homeostatic loss scales to Image Net 32 32, with a small performance gap compared to the symmetric case. 4 EXPERIMENTS In the following experiments, we used the setting of convergent recurrent neural networks (Ernoult et al., 2019), as well as the linear readout for optimizing the cross-entropy loss (Laborieux et al., 2021) (see appendix E.2). Simulations were implemented in JAX (Bradbury et al., 2018) and Flax (Heek et al., 2020) and datasets obtained through the Tensorflow Datasets API (Abadi et al., 2015). |
| Researcher Affiliation | Academia | Axel Laborieux1, Friedemann Zenke1,2 1Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland 2Faculty of Science, University of Basel, Switzerland {firstname.lastname}@fmi.ch |
| Pseudocode | No | The paper describes mathematical derivations and experimental procedures but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available on Git Hub1 and hyperparameters can be found in Appendix E.3. 1https://github.com/Laborieux-Axel/generalized-holo-ep |
| Open Datasets | Yes | To this end, we trained a two-hidden layer network with independent forward and backward connections (Fig. 1b) on the Fashion MNIST dataset (Xiao et al., 2017)... We trained this architecture using generalized h EP on CIFAR-10, CIFAR-100 (Krizhevsky, 2009) vision benchmarks... reproduced similar findings on CIFAR-100 and Image Net 32 32. |
| Dataset Splits | No | The paper mentions training on Fashion MNIST, CIFAR-10, CIFAR-100, and Image Net 32 32, and reports 'validation error' and 'validation accuracy' in tables. However, it does not explicitly state the specific train/validation/test split percentages or sample counts used for these datasets within the paper. |
| Hardware Specification | Yes | Simulations on Fashion MNIST were run on single RTX 5000 GPU... The convolutional network simulations were run on an in-house cluster consisting of 5 nodes with 4 v100 NVIDIA GPUs each, one node with 4 A100 NVIDIA GPUs, and one node with 8 A40 NVIDIA GPUs. |
| Software Dependencies | No | The paper states 'Simulations were implemented in JAX (Bradbury et al., 2018) and Flax (Heek et al., 2020) and datasets obtained through the Tensorflow Datasets API (Abadi et al., 2015).' However, it does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | Appendix E.3, titled 'HYPERPARAMETERS', provides two tables (Table 3 and Table 4) that list specific hyperparameters used for training experiments, including 'Batch size', 'Optimizer', 'Learning rate', 'Momentum', 'Epochs', 'λhomeo', 'Tfree', and 'Tnudge' for different datasets. |