The Sample Complexity of One-Hidden-Layer Neural Networks
Authors: Gal Vardi, Ohad Shamir, Nati Srebro
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study norm-based uniform convergence bounds for neural networks, aiming at a tight understanding of how these are affected by the architecture and type of norm constraint, for the simple class of scalar-valued one-hidden-layer networks, and inputs bounded in Euclidean norm. We begin by proving that in general, controlling the spectral norm of the hidden layer weight matrix is insufficient to get uniform convergence guarantees (independent of the network width), while a stronger Frobenius norm control is sufficient, extending and improving on previous work. |
| Researcher Affiliation | Academia | Gal Vardi TTI Chicago and Hebrew University galvardi@ttic.edu; Ohad Shamir Weizmann Institute of Science ohad.shamir@weizmann.ac.il; Nathan Srebro TTI Chicago nati@ttic.edu |
| Pseudocode | No | The paper presents theoretical proofs and theorems but does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | The 'Questions for Paper Analysis' section states for (3a) 'Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [N/A]'. |
| Open Datasets | No | The paper is theoretical and focuses on sample complexity bounds and proofs; it does not report experiments conducted on specific datasets, thus no dataset availability information for training is provided. |
| Dataset Splits | No | The paper is theoretical and does not conduct empirical experiments, thus no information on training, validation, or test dataset splits is provided. |
| Hardware Specification | No | The paper is purely theoretical and does not describe any experiments; therefore, no hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical and does not describe any experiments or software implementations with specific version numbers. |
| Experiment Setup | No | The paper is theoretical and does not include an experimental setup with hyperparameters or system-level training settings. |