Meta-Learning the Inductive Bias of Simple Neural Circuits
Authors: Will Dorrell, Maria Yuffa, Peter E. Latham
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our scheme by meta-learning sensible functions for linear and kernel learners, whose inductive biases are known. We now apply them to networks whose inductive bias cannot be understood analytically. Specifically: we show our method works on a challenging differentiable learner, a spiking neural network; we validate our method on a high-dimensional MNIST example; we illustrate how our tool can give normative explanations for biological circuit features, by meta-learning the impact of connectivity structures on the generalisation of a model of the fly mushroom body; and we demonstrate a method to extract animals inductive biases. |
| Researcher Affiliation | Academia | 1Gatsby Computational Neuroscience Unit, UCL. Correspondence to: William Dorrell <dorrellwec@gmail.com>. |
| Pseudocode | Yes | Algorithm 1 Meta-Learning the Learner s Inductive Bias |
| Open Source Code | Yes | We share all our code2 which should be easily adapted to networks of interest. 2Code at https://github.com/Wilbur Doz/Meta_ Learning_Inductive_Bias |
| Open Datasets | Yes | Next, we test out method on a high-dimensional input dataset. Thus far, to visualise our results, we have only considered low dimensional input data. We demonstrate that our method continues to work in high-dimensions by applying it to a dataset made of the 0 and 1 MNIST digits (Le Cun, 1998). |
| Dataset Splits | No | The paper mentions training and testing on data but does not provide specific train/validation/test split percentages, sample counts, or explicit splitting methodology for its experiments. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory) used for running the experiments. |
| Software Dependencies | No | Our tool is flexible: by taking gradients through the training procedure we can meta-learn inductive biases for networks trained using Py Torch, for example. We share all our code2 which should be easily adapted to networks of interest. (No specific PyTorch version mentioned) and while some referenced libraries have years, explicit version numbers for dependencies are missing. |
| Experiment Setup | No | The paper describes the general training process, network architectures, and loss functions (e.g., 'we used 30' datapoints for training), but lacks specific numerical hyperparameter values such as learning rates, batch sizes, or the total number of training epochs/iterations for the meta-learner and learners. |