Artificial Neuronal Ensembles with Learned Context Dependent Gating
Authors: Matthew James Tilley, Michelle Miller, David Freedman
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the ability of this method to alleviate catastrophic forgetting on continual learning benchmarks. When the new regularization terms are included in the model along with Elastic Weight Consolidation (EWC) it achieves better performance on the benchmark permuted MNIST than with EWC alone. The benchmark rotated MNIST demonstrates how similar tasks recruit similar neurons to the artificial neuronal ensemble. |
| Researcher Affiliation | Academia | Matthew J. Tilley, Michelle Miller & David J. Freedman Department of Neurobiology, University of Chicago Chicago, IL 60637, USA {mjtilley, mcmiller1, dfreedman}@uchicago.edu |
| Pseudocode | No | The paper describes the mathematical formulation of its method and regularization terms but does not include a pseudocode block or an algorithm section. |
| Open Source Code | No | The paper does not provide an explicit statement or link regarding the availability of its source code. |
| Open Datasets | Yes | We trained the models on two benchmark tasks, permuted MNIST (Goodfellow et al., 2013) and rotated MNIST. Permuted MNIST is a task in which the pixels of the traditional MNIST images (Lecun et al., 1998) are shuffled to create new tasks. |
| Dataset Splits | No | The paper states 'For the training and test set in each task, there are 60,000 and 10,000 images respectively,' but does not explicitly detail a validation set split or its size. |
| Hardware Specification | Yes | All models were trained in Pytorch (Paszke et al., 2019) on NVIDIA s Ge Force GTX 1080 Tis. |
| Software Dependencies | No | The paper mentions 'Pytorch (Paszke et al., 2019)' as the framework used, but does not specify its version number or any other software dependencies with their versions. |
| Experiment Setup | Yes | The model was trained on 28x28 permuted MNIST images across 50 tasks (i.e. 50 unique ways of shuffling or rotating the image), each with 20 epochs. The models are trained with backpropagation with a learning rate of .001 along with batch sizes of 256. |