On the Inherent Regularization Effects of Noise Injection During Training
Authors: Oussama Dhifallah, Yue Lu
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical results corroborate our asymptotic predictions, showing that they are accurate even in moderate problem dimensions. Our theoretical predictions are based on a new correlated Gaussian equivalence conjecture that generalizes recent results in the study of random feature models. |
| Researcher Affiliation | Academia | Oussama Dhifallah 1 Yue M. Lu 1 1O. Dhifallah and Y. M. Lu are with the John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA.. |
| Pseudocode | No | No pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | No | The paper does not provide any explicit statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | In Figure 6(a), we consider the Semeion Handwritten Digit Data Set downloaded from the Machine Learning Repository . |
| Dataset Splits | No | The paper does not explicitly provide training/test/validation dataset splits, specific percentages, or detailed splitting methodologies, although it discusses training and generalization errors. |
| Hardware Specification | No | The paper mentions numerical simulations and experiments but does not provide any specific details about the hardware (e.g., GPU/CPU models, memory) used to run them. |
| Software Dependencies | No | The paper does not provide specific software dependencies (e.g., library names with version numbers) needed to replicate the experiments. |
| Experiment Setup | Yes | Figure 1. Solid line: Theoretical predictions. Circle: Numerical simulations for (3). Black cross: Numerical simulations for (6). ϕ( ) is the sign function with probability θ of flipping the sign. bϕ( ) and σ( ) are the sign function. We set p = 500, = 0.5, α = n/p = 2, θ = 0.1, λ = 10 5. F has independent Gaussian components with zero mean and variance 1/p. |