Curriculum learning as a tool to uncover learning principles in the brain
Authors: Daniel R. Kepple, Rainer Engelken, Kanaka Rajan
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using recurrent neural networks (RNNs) and models of common experimental neuroscience tasks, we demonstrate that curricula can be used to differentiate learning principles using target-based and a representation-based loss functions as use cases. |
| Researcher Affiliation | Academia | Daniel R Kepple Friedman Brain Institute Icahn School of Medicine at Mount Sinai New York, NY 10029, USA Rainer Engelken Department of Neuroscience, Zuckerman Institute Columbia University New York, NY, 10027, USA Kanaka Rajan Friedman Brain Institute Icahn School of Medicine at Mount Sinai New York, NY 10029, USA |
| Pseudocode | No | The paper describes the RNN model equations and learning rules in text and mathematical formulas but does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code or links to a code repository. |
| Open Datasets | No | The paper describes generating its own task-relevant input data based on models of neuroscience tasks (e.g., 'Cue start times for each channel are Poisson distributed'). It does not use a publicly available dataset with concrete access information for training. |
| Dataset Splits | No | The paper mentions a 'test set' but does not explicitly describe a separate 'validation' set or specific train/test/validation dataset splits or cross-validation methodology for reproduction. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments (e.g., GPU/CPU models, memory, or cloud instance types). |
| Software Dependencies | No | The paper mentions the use of the Euler method and Adam optimizer but does not specify any software environments, libraries, or programming languages with their version numbers. |
| Experiment Setup | Yes | We use neural networks with N = 350 recurrently connected, continuous, firing rate based, leaky integrating model neurons. τ is the time constant of the neuron, here, 10ms. Recurrent weights W rec are initialized i.i.d from a Gaussian with mean 0 and variance g2 N . We set, g = 1. Input W inp and readout weights W out are each drawn from a uniform distribution from -1 to 1. Internal states, X, are initialized from a Gaussian (mean 0, variance 1). Gradients are calculated and accumulated at every time step in parallel trials presented in batch sizes of 32. Weights are updated after each trial using Adam with β0 = 0.9 and β1 = 0.999 (Kingma & Ba, 2014). Learning rates are chosen proportionally to average weight initialization; 0.01 for readout or output weights, W out and representational weights W rep, and 0.0003 for recurrent weights, W rec. |