Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Identifying Learning Rules From Neural Network Observables
Authors: Aran Nayebi, Sanjana Srivastava, Surya Ganguli, Daniel L. Yamins
NeurIPS 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Simulating idealized neuroscience experiments with artificial neural networks, we generate a large-scale dataset of learning trajectories of aggregate statistics measured in a variety of neural network architectures, loss functions, learning rule hyperparameters, and parameter initializations. We then take a discriminative approach, training linear and simple non-linear classifiers to identify learning rules from features based on these observables. |
| Researcher Affiliation | Academia | 1Neurosciences Ph.D. Program, Stanford University 2Department of Computer Science, Stanford University 3Department of Applied Physics, Stanford University 4Department of Psychology, Stanford University 5Wu Tsai Neurosciences Institute, Stanford University |
| Pseudocode | No | The paper describes methods and procedures in narrative text and refers to figures, but it does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | 3https://github.com/neuroailab/lr-identify |
| Open Datasets | Yes | We consider the tasks of supervised 1000-way Image Net categorization [Deng et al., 2009]... and supervised ten-way CIFAR-10 categorization [Krizhevsky, 2010]. |
| Dataset Splits | Yes | test accuracy of each classifier, with mean and s.e.m. across ten category-balanced 75%/25% train/test splits, using the observable measures in 3. |
| Hardware Specification | Yes | We thank the Google Tensor Flow Research Cloud (TFRC) team for generously providing TPU hardware resources for this project. |
| Software Dependencies | No | The provided text does not specify any software names with version numbers (e.g., Python 3.8, PyTorch 1.9, TensorFlow 2.x) that would allow for replication of the experimental environment. |
| Experiment Setup | Yes | Learning hyperparameters for each model under a given learning rule category are the Cartesian product of three settings of batch size (128, 256, and 512)... All model training details can be found in Appendix A. |