Dimension-free deterministic equivalents and scaling laws for random feature regression
Authors: Leonardo Defilippis, Bruno Loureiro, Theodor Misiakiewicz
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically validate its predictions on various real and synthetic datasets. |
| Researcher Affiliation | Academia | Département d Informatique École Normale Supérieure PSL & CNRS Department of Statistics and Data Science Yale University |
| Pseudocode | Yes | Algorithm 1 Empirical diagonalization |
| Open Source Code | No | We judge the code is too simple to be released, and that we give enough information for the reproducibility of the numerical plots. All data sets used in the numerical experiments are either synthetic or open source. |
| Open Datasets | Yes | We performed numerical simulations sampling the training data from the MNIST data set Lecun et al. [1998] and the Fashion MNIST data set Xiao et al. [2017] |
| Dataset Splits | No | No explicit training/validation/test dataset splits (e.g., percentages, counts, or predefined split citations) are provided. The paper mentions using 'training data' and evaluating 'test error' but without detailing the splitting methodology. |
| Hardware Specification | No | No specific hardware details (e.g., CPU/GPU models, memory amounts, or cloud instance types) are explicitly provided for running the experiments. The NeurIPS checklist also indicates no such information is provided. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., libraries, frameworks, or solvers) are explicitly mentioned for replicating the experiments. |
| Experiment Setup | Yes | To solve Equations 18 and 19 numerically, the following approach has been employed. From equation 19, s = 1/n ... Substituting this expression in equation 18, we obtain... The parameters ν1 and ν2 have been computed by iterating... until a chosen tolerance ϵ was reached. / In Figure 7 (right),... learning rate η = 10-2, before training the second layer with regularization strength λ = 10-4. |