Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization
Authors: Ravi Francesco Srinivasan, Francesca Mignacco, Martino Sorbaro, Maria Refinetti, Avi Cooper, Gabriel Kreiman, Giorgia Dellaferrera
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | FORWARD LEARNING WITH TOP-DOWN FEEDBACK: EMPIRICAL AND ANALYTICAL CHARACTERIZATION, Then, we compare different versions of forward-only algorithms, focusing on the Forward-Forward and PEPITA frameworks, Table 1: Biological properties and test accuracy [%] on MNIST achieved by a non-exhaustive selection of bio-inspired alternatives to BP., Fig. 2b shows our theoretical prediction for the generalization error as a function of time, compared to numerical simulations of the PEPITA and AFA algorithms. |
| Researcher Affiliation | Collaboration | 1 IBM Research Europe, Zurich, 2 ETH Zurich, 3 Joseph Henry Laboratories of Physics, Princeton University, 4 Initiative for the Theoretical Sciences, Graduate Center, City University of New York, 5 Institute of Neuroinformatics, University of Zurich and ETH Zurich, 6 AI Center, ETH Zurich, 7 Laboratoire de Physique de l Ecole Normale Sup erieure, Universit e PSL, CNRS, Sorbonne Universit e, Universit e Paris-Diderot, Sorbonne Paris Cit e, 8 Ide PHICS laboratory, Ecole F ed erale Polytechnique de Lausanne (EPFL), 9 Massachusetts Institute of Technology, 10 Center for Brains, Minds and Machines, 11 Children s Hospital, Harvard Medical School |
| Pseudocode | Yes | The pseudocode detailing the algorithm is provided in Appendix A., Algorithm S1 Implementation of PEPITA, Algorithm S2 Impl. of PEPITA-time-local |
| Open Source Code | Yes | Experiments were run on Nvidia V100 GPUs, using custom Python code available here. |
| Open Datasets | Yes | MNIST (Le Cun & Cortes, 2010) and CIFAR-10 (Krizhevsky et al., a), CIFAR-100 (Krizhevsky et al., b) |
| Dataset Splits | No | The paper mentions 'test accuracy' and 'generalization error' but does not explicitly provide specific dataset split information (percentages, sample counts, or citations to predefined splits with full details) for training, validation, and test sets. |
| Hardware Specification | Yes | Experiments were run on Nvidia V100 GPUs |
| Software Dependencies | No | The paper mentions 'custom Python code' but does not provide specific version numbers for Python or any other software dependencies such as libraries or frameworks. |
| Experiment Setup | Yes | The hyperparameters are found through grid search and are reported in Table S3., Table S3: 1-hidden-layer network architectures and settings used in the experiments. The nonlinearity is Re LU for all algorithms except DRTP, for which is tanh., Table S4: 2-, 3-hidden-layer network architectures and settings used in the experiments. The nonlinearity is Re LU for all algorithms except DRTP, for which is tanh. |