Identifying Learning Rules From Neural Network Observables
Authors: Aran Nayebi, Sanjana Srivastava, Surya Ganguli, Daniel L. Yamins
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Simulating idealized neuroscience experiments with artificial neural networks, we generate a large-scale dataset of learning trajectories of aggregate statistics measured in a variety of neural network architectures, loss functions, learning rule hyperparameters, and parameter initializations. We then take a discriminative approach, training linear and simple non-linear classifiers to identify learning rules from features based on these observables. |
| Researcher Affiliation | Academia | 1Neurosciences Ph.D. Program, Stanford University 2Department of Computer Science, Stanford University 3Department of Applied Physics, Stanford University 4Department of Psychology, Stanford University 5Wu Tsai Neurosciences Institute, Stanford University |
| Pseudocode | No | The paper describes methods and procedures in narrative text and refers to figures, but it does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | 3https://github.com/neuroailab/lr-identify |
| Open Datasets | Yes | We consider the tasks of supervised 1000-way Image Net categorization [Deng et al., 2009]... and supervised ten-way CIFAR-10 categorization [Krizhevsky, 2010]. |
| Dataset Splits | Yes | test accuracy of each classifier, with mean and s.e.m. across ten category-balanced 75%/25% train/test splits, using the observable measures in 3. |
| Hardware Specification | Yes | We thank the Google Tensor Flow Research Cloud (TFRC) team for generously providing TPU hardware resources for this project. |
| Software Dependencies | No | The provided text does not specify any software names with version numbers (e.g., Python 3.8, PyTorch 1.9, TensorFlow 2.x) that would allow for replication of the experimental environment. |
| Experiment Setup | Yes | Learning hyperparameters for each model under a given learning rule category are the Cartesian product of three settings of batch size (128, 256, and 512)... All model training details can be found in Appendix A. |