Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processes
Authors: Jun Yang, Shengyang Sun, Daniel M. Roy
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | The goal of this paper is to extend this bridge between Rademacher complexity and state-of-the-art PACBayesian theory. We first demonstrate that one can match the fast rate of Catoni s PAC-Bayes bounds [8] using shifted Rademacher processes [27, 43, 44]. We then derive a new fast-rate PAC-Bayes bound in terms of the flatness of the empirical risk surface on which the posterior concentrates. Our analysis establishes a new framework for deriving fast-rate PAC-Bayes bounds and yields new insights on PAC-Bayesian theory. |
| Researcher Affiliation | Academia | Jun Yang Department of Statistical Sciences University of Toronto, Vector Institute jun@utstat.toronto.edu Shengyang Sun Department of Computer Science University of Toronto, Vector Institute ssy@cs.toronto.edu Daniel M. Roy Department of Statistical Sciences University of Toronto, Vector Institute droy@utstat.toronto.edu |
| Pseudocode | No | The paper focuses on mathematical derivations and theoretical framework, and does not contain any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link regarding the availability of open-source code for the described methodology. |
| Open Datasets | No | The paper is theoretical and does not conduct experiments on specific datasets, thus no training dataset information is provided. |
| Dataset Splits | No | The paper is theoretical and does not involve empirical experiments with dataset splits, thus no validation split information is provided. |
| Hardware Specification | No | The paper is theoretical and does not describe any experimental hardware specifications. |
| Software Dependencies | No | The paper is theoretical and does not list any specific software dependencies with version numbers. |
| Experiment Setup | No | The paper is theoretical and does not describe an experimental setup with hyperparameters or training configurations. |