Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization
Authors: Mufan Li, Mihai Nica, Dan Roy
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using simulations, we show that the SDE closely matches the distribution of the random covariance matrix of ๏ฌnite networks. Additionally, we recover an if-and-only-if condition for exploding and vanishing norms of large shaped networks based on the activation function. We provide simulations to verify theoretical predictions and help interpret properties of real DNNs. See Figures 1 and 2 and supplemental simulations in Appendix F. |
| Researcher Affiliation | Academia | Mufan (Bill) Li University of Toronto, Vector Institute Mihai Nica University of Guelph, Vector Institute Daniel M. Roy University of Toronto, Vector Institute |
| Pseudocode | No | No pseudocode or algorithm blocks found. |
| Open Source Code | Yes | All of our simulations (at initialization only) are contained the ๏ฌle Correlation.ipynb. |
| Open Datasets | No | The paper describes simulations, not experiments on a publicly available dataset. No concrete access information for a dataset used for training or evaluation is provided. |
| Dataset Splits | No | No specific dataset splits (training, validation, test) are mentioned, as the paper focuses on simulations rather than using a standard dataset. |
| Hardware Specification | No | Our simulations were small enough that it did not require GPUs. No specific hardware details (e.g., CPU model, memory) are provided. |
| Software Dependencies | No | No specific software dependencies with version numbers are mentioned. |
| Experiment Setup | Yes | Simulation details: n = d = 150, ฯ0 = 0.3, 213 samples for each. In right column: c+ = 0, c = 1, DE step size 1e 2. |