Noisy Interpolation Learning with Shallow Univariate ReLU Networks
Authors: Nirmit Joshi, Gal Vardi, Nathan Srebro
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We provide the first rigorous analysis of the overfitting behavior of regression with minimum norm (ℓ2 of weights), focusing on univariate two-layer Re LU networks. We show overfitting is tempered (with high probability) when measured with respect to the L1 loss, but also show that the situation is more complex than suggested by Mallinar et al., and overfitting is catastrophic with respect to the L2 loss, or when taking an expectation over the training set. |
| Researcher Affiliation | Academia | Nirmit Joshi TTI-Chicago nirmit@ttic.edu Gal Vardi TTI-Chicago and Hebrew University galvardi@ttic.edu Nathan Srebro TTI-Chicago nati@ttic.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statements or links indicating the availability of open-source code for the described methodology. |
| Open Datasets | No | The paper considers a theoretical data model: 'We consider a noisy distribution D over [0, 1] R: x Uniform([0, 1]) and y = f (x) + ϵ with ϵ independent of x, where x is uniform for simplicity and concreteness'. It does not provide access information for a publicly available dataset. |
| Dataset Splits | No | The paper is theoretical and analyzes properties of the interpolator under a defined noise model. It does not describe training, validation, or test dataset splits in the context of empirical experiments. |
| Hardware Specification | No | The paper is theoretical and does not mention any specific hardware used for computations or analysis. |
| Software Dependencies | No | The paper is theoretical and does not list any specific software dependencies with version numbers for reproducibility. While 'Python errors' are mentioned in acknowledgments, this is not a specification of dependencies for the research itself. |
| Experiment Setup | No | The paper is theoretical and does not describe any experimental setup details such as hyperparameters or system-level training settings. |