Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Universality of Kernel Random Matrices and Kernel Regression in the Quadratic Regime
Authors: Parthe Pandit, Zhichao Wang, Yizhe Zhu
JMLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we extend the study of kernel regression to the quadratic asymptotic regime, where n d2. In this regime, we demonstrate that a broad class of inner-product kernels exhibits behavior similar to a quadratic kernel. Specifically, we establish an operator norm approximation bound for the difference between the original kernel random matrix and a quadratic kernel random matrix with additional correction terms compared to the Taylor expansion of the kernel functions. The approximation works for general data distributions... This new approximation is utilized to obtain a limiting spectral distribution... and characterize the precise asymptotic training and test errors for KRR... Our proof techniques combine moment methods, Wick s formula, orthogonal polynomials, and resolvent analysis of random matrices with correlated entries. (Abstract) In this section, we provide several simulations to illustrate our theoretical results. ... (a) Σ = Id, n = 18000 and d = 200. (b) Σ = Σ0, n = 18000 and d = 200. (c) Σ = Id, n = 25250 and d = 150. (d) Σ = Σ0, n = 25250 and d = 150. Figure 3: Spectral distributions for kernel function f(x) = x2 + x with isotropic and anisotropic Gasussian datasets. The red curves are given by the limiting spectral distribution obtained from Theorem 2. |
| Researcher Affiliation | Academia | Parthe Pandit EMAIL Center for Machine Intelligence and Data Science Indian Institute of Technology Bombay Mumbai, Maharashtra 400076, India Zhichao Wang EMAIL International Computer Science Institute Department of Statistics University of California, Berkeley Berkeley, CA 94720, USA Yizhe Zhu EMAIL Department of Mathematics University of Southern California Los Angeles, CA 90089, USA |
| Pseudocode | No | The paper focuses on theoretical derivations, proofs, and mathematical analysis. There are no sections titled 'Pseudocode' or 'Algorithm,' nor are there any structured code-like blocks describing a procedure. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing code, nor does it provide any links to a code repository. The license information 'c 2025 Parthe Pandit, Zhichao Wang, Yizhe Zhu. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v26/24-1533.html' refers to the paper's content license, not source code. |
| Open Datasets | No | The paper describes using 'Gaussian data' and 'isotropic/anisotropic Gaussian datasets' for simulations (e.g., Section 3, Figure 3), which are synthetically generated based on distributions. It does not mention any specific publicly available datasets by name (like CIFAR-10) or provide links/citations to external data sources. |
| Dataset Splits | No | The paper does not specify any dataset splits like 'training/test/validation percentages' or sample counts. While it mentions 'random training data' and 'new data point (x, f (x)) where x Rd is independent with all training data points xi' in the context of generalization error, these are conceptual definitions within its theoretical framework rather than concrete experimental splits. The simulations in Section 3 use generated Gaussian data but do not detail splits. |
| Hardware Specification | No | The paper discusses numerical simulations in Section 3 but does not provide any details about the hardware used to run these simulations (e.g., specific GPU or CPU models, memory). |
| Software Dependencies | No | The paper does not mention any specific software dependencies with version numbers, such as programming languages, libraries, or frameworks (e.g., Python, PyTorch, TensorFlow). |
| Experiment Setup | Yes | In Figure 5(a), we present a simulation for the test losses of KRR... We fix d = 160, and use isotropic Gaussian data, polynomial kernel f(x) = (1 + x)2, λ = 0.01, and σϵ = 0.5. This simulation also demonstrates the double descent phenomenon. |