On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decay
Authors: Yicheng Li, haobo Zhang, Qian Lin
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we consider numerical experiments on a toy model to verify our theory. For some f , we generate data from the model y = f (x) + ε where ε N(0, 0.05) and perform KRR with λ = cn θ for different θ s with some fixed constant c. Then, we numerically compute the variance, bias and excess risk by Simpson s formula with N n nodes. Repeating the experiment for n ranged in 1000 to 5000, we can estimate the convergence rate r by a logarithmic least-squares log err = r log n + b on the values (variance, bias and excess risk). The results are collected in Table 1 on page 10. |
| Researcher Affiliation | Academia | Yicheng Li, Haobo Zhang Center for Statistical Science, Department of Industrial Engineering Tsinghua University, Beijing, China {liyc22,zhang-hb21}@mails.tsinghua.edu.cn Qian Lin Center for Statistical Science, Department of Industrial Engineering Tsinghua University, Beijing, China qianlin@tsinghua.edu.cn Qian Lin also affiliates with Beijing Academy of Artificial Intelligence, Beijing, China |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any statements about releasing open-source code or provide links to a code repository for the described methodology. |
| Open Datasets | No | The paper uses synthetic data generated from a toy model ("Let us consider the kernel k(x, y) = min(x, y) and x U[0, 1]... For some f , we generate data from the model y = f (x) + ε"), rather than a publicly available dataset. |
| Dataset Splits | No | The paper does not explicitly provide training/validation/test dataset splits. It generates synthetic data to numerically verify theoretical results, stating: "Repeating the experiment for n ranged in 1000 to 5000." |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments. |
| Software Dependencies | No | The paper does not list specific software components with their version numbers that would be needed to reproduce the experiment. |
| Experiment Setup | Yes | For some f , we generate data from the model y = f (x) + ε where ε N(0, 0.05) and perform KRR with λ = cn θ for different θ s with some fixed constant c. ... Note that for each setting, we tried different c s in the regularization parameter λ = cn θ and show the curves under the best choice of c (c = 0.005). |