Benign overfitting in leaky ReLU networks with moderate input dimension
Authors: Kedar Karhadkar, Erin George, Michael Murray, Guido F. Montufar, Deanna Needell
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To further support our theory, we train shallow neural networks on the data model described in Definition 2.1 and record the numerical results. Scripts to reproduce these experiments can be found at https://github.com/kedar2/benign_overfitting. These experiments were run on the CPU of a Mac Book Pro M2 with 8GB of RAM. |
| Researcher Affiliation | Academia | Kedar Karhadkar1 Erin George1 Michael Murray1 Guido Montúfar12 Deanna Needell1 {kedar,egeo,mmurray,montufar,deanna}@math.ucla.edu 1UCLA 2Max Planck Institute for Mathematics in the Sciences Equal contribution |
| Pseudocode | No | The paper contains mathematical equations and descriptions of algorithms, but no structured pseudocode blocks or algorithm boxes are presented. |
| Open Source Code | Yes | Scripts to reproduce these experiments can be found at https://github.com/kedar2/benign_overfitting. |
| Open Datasets | No | 2.1 Data model We study data generated as per the following data model. Definition 2.1. Suppose d, n, k N, (0, 1) and v Sd 1. If (X, y, y, x, y) D(d, n, k, , v) then 1. X Rn d is a random matrix whose rows, which we denote xi, satisfy xi = yiv + 1 ni, where ni N(0d, 1 d(Id vv T )) are mutually i.i.d.. |
| Dataset Splits | Yes | Parameter settings: = 0.1, = 5/n, m = 64, k = 0.1n, number of trials = 5, size of validation sample = 1000. |
| Hardware Specification | Yes | These experiments were run on the CPU of a Mac Book Pro M2 with 8GB of RAM. |
| Software Dependencies | No | The paper describes the learning algorithm (gradient descent with hinge loss) but does not specify any particular software libraries or frameworks with version numbers (e.g., PyTorch, TensorFlow). |
| Experiment Setup | Yes | Parameter settings: = 0.1, = 5/n, m = 64, k = 0.1n, number of trials = 5, size of validation sample = 1000. |