Exact marginal prior distributions of finite Bayesian neural networks
Authors: Jacob Zavatone-Veth, Cengiz Pehlevan
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Figure 1: Priors of deep linear networks of depths d = 2, 3, and 4. In each panel, the prior density is plotted only for positive values of the output preactivation hd, as it is symmetric about zero. For each depth, all hidden layers are of the same width n, which is indicated by line color. The black line indicates the Gaussian infinite-width limit discussed in 4.3. Thick lines show the exact priors, while thin jagged lines show experimental estimates from 10^8 examples. Further details on the numerical methods used to generate these figures are provided in Appendix E. |
| Researcher Affiliation | Academia | Jacob A. Zavatone-Veth Department of Physics Harvard University Cambridge, MA 02138 jzavatoneveth@g.harvard.edu Cengiz Pehlevan John A. Paulson School of Engineering and Applied Sciences Harvard University Cambridge, MA 02138 cpehlevan@seas.harvard.edu |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks; it primarily presents mathematical derivations and formulas. |
| Open Source Code | No | The paper does not provide any explicit statements or links indicating the release of its source code. |
| Open Datasets | No | The paper does not use or reference a publicly available dataset for training in the traditional machine learning sense. Its 'experimental estimates' are generated by numerical sampling from prior distributions. |
| Dataset Splits | No | The paper performs numerical sampling to validate its theoretical results ('experimental estimates from 10^8 examples') but does not describe training, validation, or test dataset splits in the conventional machine learning sense, as it does not train models on typical datasets. |
| Hardware Specification | No | The computations in this paper were performed using the Harvard University FAS Division of Science Research Computing Group s Cannon HPC cluster. While a specific cluster name is given, no detailed hardware specifications (e.g., CPU/GPU models, memory) are provided. |
| Software Dependencies | No | Appendix E mentions the use of 'scipy.special.gamma' and 'scipy.special.kv' for numerical computation, implying Python and SciPy are used, but no specific version numbers for any software dependencies are provided. |
| Experiment Setup | Yes | For Figures 1, 3, and 4, we generate experimental estimates by sampling from the prior distribution... We take 10^8 samples for all reported curves, which is sufficient to obtain visually indistinguishable estimates... We compute the exact priors for a grid of 1000 equally spaced points... The numerical integral in (A.8) is computed using a Gaussian quadrature rule with 1000 points. |