Asymptotics of representation learning in finite Bayesian neural networks
Authors: Jacob Zavatone-Veth, Abdulkadir Canatar, Ben Ruben, Cengiz Pehlevan
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that our theory yields quantitatively accurate predictions for the result of numerical experiment for tractable linear network architectures, and qualitatively accurate predictions for deep nonlinear networks, where quantitative analytical predictions are intractable. |
| Researcher Affiliation | Academia | Jacob A. Zavatone-Veth1,2, Abdulkadir Canatar1,2, Benjamin S. Ruben3, Cengiz Pehlevan2,4 1Department of Physics, 2Center for Brain Science, 3Biophysics Graduate Program, 4John A. Paulson School of Engineering and Applied Sciences Harvard University Cambridge, MA 02138 {jzavatoneveth,canatara,benruben}@g.harvard.edu cpehlevan@seas.harvard.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper mentions using the 'Neural Tangents library [33]' and PyTorch [31], but it does not provide a link or statement for the source code specific to the authors' methodology or implementation for this paper. |
| Open Datasets | Yes | trained on the MNIST dataset of handwritten digit images [32]. [32] Yann Le Cun, Corinna Cortes, and CJ Burges. MNIST handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010. |
| Dataset Splits | No | The paper mentions using a certain number of MNIST images for training (e.g., '5000 MNIST images' or '1000 MNIST images') but does not specify how these images were split into training, validation, and test sets, or if standard splits were used without explicit mention. |
| Hardware Specification | No | The paper does not specify any particular hardware (CPU, GPU models, etc.) used for running the experiments. It only mentions the software libraries used. |
| Software Dependencies | Yes | Using Langevin sampling [30, 31]... Neural Tangents library [33]. [31] Pytorch: An imperative style, highperformance deep learning library. In H. Wallach... 2019. [33] Neural tangents: Fast and easy infinite neural networks in python. In International Conference on Learning Representations, 2020. |
| Experiment Setup | Yes | We provide a detailed discussion of our numerical methods in Appendix I. For the numerical experiments, we used a Langevin step size of 0.1 for 10^5 steps with a batch size of 16 for fully connected networks and 64 for convolutional networks... The number of hidden layers is varied between 2 and 3, and the width n of each layer is varied from 16 to 256 for fully connected networks and from 16 to 128 for convolutional networks. |