Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms
Authors: Dimitri Meunier, Zikai Shen, Mattes Mollenhauer, Arthur Gretton, Zhu Li
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study theoretical properties of a broad class of regularized algorithms with vector-valued output. These spectral algorithms include kernel ridge regression, kernel principal component regression and various implementations of gradient descent. Our contributions are twofold. First, we rigorously confirm the so-called saturation effect for ridge regression with vector-valued output by deriving a novel lower bound on learning rates; this bound is shown to be suboptimal when the smoothness of the regression function exceeds a certain level. Second, we present an upper bound on the finite sample risk for general vector-valued spectral algorithms, applicable to both well-specified and misspecified scenarios (where the true regression function lies outside of the hypothesis space), and show that this bound is minimax optimal in various regimes. All of our results explicitly allow the case of infinite-dimensional output variables, proving consistency of recent practical applications. |
| Researcher Affiliation | Collaboration | Dimitri Meunier Gatsby Computational Neuroscience Unit University College London dimitri.meunier.21@ucl.ac.uk Zikai Shen Department of Statistical Science University College London zikai.shen.22@ucl.ac.uk Mattes Mollenhauer Merantix Momentum mattes.mollenhauer@merantix-momentum.com Arthur Gretton Gatsby Computational Neuroscience Unit University College London arthur.gretton@gmail.com Zhu Li Department of Mathematics Imperial College London zli12@ic.ac.uk |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. Its content is purely theoretical with mathematical derivations. |
| Open Source Code | No | The paper does not include any statement about releasing source code or provide links to a code repository. |
| Open Datasets | No | The paper is theoretical and does not involve experiments or the use of datasets for training. Thus, it does not provide access information for a training dataset. |
| Dataset Splits | No | The paper is theoretical and does not describe any experiments that would require dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper is theoretical and does not describe any experiments, thus no hardware specifications are provided. |
| Software Dependencies | No | The paper is theoretical and does not describe any experiments or implementations, thus no software dependencies with version numbers are listed. |
| Experiment Setup | No | The paper is theoretical and focuses on mathematical proofs and derivations. It does not describe any experimental setup, hyperparameters, or training configurations. |