Fast learning rates with heavy-tailed losses
Authors: Vu C. Dinh, Lam S. Ho, Binh Nguyen, Duy Nguyen
NeurIPS 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study fast learning rates when the losses are not necessarily bounded and may have a distribution with heavy tails. To enable such analyses, we introduce two new conditions: (i) the envelope function supf F |ℓ f|, where ℓis the loss function and F is the hypothesis class, exists and is Lr-integrable, and (ii) ℓsatisfies the multi-scale Bernstein s condition on F. Under these assumptions, we prove that learning rate faster than O(n 1/2) can be obtained and, depending on r and the multi-scale Bernstein s powers, can be arbitrarily close to O(n 1). We then verify these assumptions and derive fast learning rates for the problem of vector quantization by k-means clustering with heavy-tailed distributions. The analyses enable us to obtain novel learning rates that extend and complement existing results in the literature from both theoretical and practical viewpoints. |
| Researcher Affiliation | Academia | Vu Dinh1 Lam Si Tung Ho2 Duy Nguyen3 Binh T. Nguyen4 1Program in Computational Biology, Fred Hutchinson Cancer Research Center 2Department of Biostatistics, University of California, Los Angeles 3Department of Statistics, University of Wisconsin-Madison 4Department of Computer Science, University of Science, Vietnam |
| Pseudocode | No | The paper describes mathematical frameworks and proofs but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any statements about making source code available, nor does it provide links to a code repository. |
| Open Datasets | No | This paper is theoretical and does not involve empirical experiments with datasets. Therefore, there is no mention of publicly available or open datasets used for training. |
| Dataset Splits | No | This paper is theoretical and does not involve empirical experiments with datasets. Therefore, it does not specify any training/validation/test dataset splits. |
| Hardware Specification | No | This paper is theoretical and does not describe any empirical experiments, thus it does not mention specific hardware used for running experiments. |
| Software Dependencies | No | This paper is theoretical and does not describe any empirical experiments, thus it does not mention specific software dependencies with version numbers. |
| Experiment Setup | No | This paper is theoretical and focuses on mathematical proofs and analysis. It does not describe an experimental setup, hyperparameters, or system-level training settings. |