Learning in the Presence of Low-dimensional Structure: A Spiked Random Matrix Perspective
Authors: Jimmy Ba, Murat A. Erdogdu, Taiji Suzuki, Zhichao Wang, Denny Wu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we empirically validate the theoretical results presented in previous sections. We construct target functions f with a link of varying degree p and information exponent k and experimentally compute the prediction risk of KRR and that of a two-layer NN. |
| Researcher Affiliation | Collaboration | 1University of Toronto, 2Vector Institute, 3x AI, 4University of Tokyo, 5RIKEN AIP, 6University of California San Diego, 7New York University, 8Flatiron Institute |
| Pseudocode | Yes | This procedure is summarized as follows (see Algorithm 1 in Appendix for more details). |
| Open Source Code | No | The paper does not contain any explicit statements or links indicating that the source code for the described methodology is publicly available. |
| Open Datasets | No | We consider the problem of learning a single-index target function f : Rd R under the spiked covariance data: f (x) = σ 1 1+θ x, µ , x N(0, Id + θµµ ), θ dβ for β [0, 1),... where the goal is to estimate the target function (teacher) f , which is a single-index model depending on the signal direction µ Rd, and σ : R R is an unknown link function. |
| Dataset Splits | No | The paper operates within a 'proportional asymptotic limit' (n, d -> infinity, n/d -> psi (0, infinity)) and uses synthetically generated data. It does not mention explicit train/validation/test splits by percentages or sample counts, nor does it refer to predefined splits from a standard public dataset. While it evaluates prediction risk, the concept of a dataset split in the conventional empirical sense is not detailed. |
| Hardware Specification | No | The paper does not specify any hardware details (e.g., GPU models, CPU types, or memory) used for running the experiments described in Section 5. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used in its experiments. |
| Experiment Setup | Yes | We set ψ = 5. For KRR we use the Gaussian RBF kernel and λ = 10^-2. For two-layer NN, we set η = 1/√N. |