Minimizing Control for Credit Assignment with Strong Feedback
Authors: Alexander Meulemans, Matilde Tristany Farinha, Maria R. Cervera, João Sacramento, Benjamin F. Grewe
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We complement our theoretical results with experiments on standard computer-vision benchmarks, showing competitive performance to backpropagation as well as robustness to noise. Overall, our work presents a fundamentally novel view of learning as control minimization, while sidestepping biologically unrealistic assumptions. |
| Researcher Affiliation | Academia | 1Institute of Neuroinformatics, University of Z urich and ETH Z urich, Switzerland. Correspondence to: Alexander Meulemans <ameulema@ethz.ch>. |
| Pseudocode | Yes | Algorithm 1 Pseudocode for the Strong-DFC algorithm on a single input sample |
| Open Source Code | Yes | Source code for all experiments is available at: https:// github.com/mariacer/strong_dfc. |
| Open Datasets | Yes | We confirm the above results for the idealized setting on the MNIST dataset (Le Cun et al., 2010) (Table 1). Here we also observe that both the original training loss L and the surrogate loss H reach low values, which in turn translates into high testing accuracy. |
| Dataset Splits | Yes | We optimize the hyperparameters of each method independently, for best performance on a validation set of 5000 datapoints. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. It mentions 'conventional deep learning hardware' but no specifics. |
| Software Dependencies | No | The paper does not provide specific software dependency versions (e.g., Python 3.8, PyTorch 1.9). It describes the algorithms and their implementation but omits specific software versions. |
| Experiment Setup | Yes | We optimize the hyperparameters of each method independently, for best performance on a validation set of 5000 datapoints. We use BP to train a noiseless network, and DFC to train both a noiseless and a noisy network, where we add noise to the dynamics according to Eq. (8). |