Generalization Bounds for Neural Networks via Approximate Description Length
Authors: Amit Daniely, Elad Granot
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We investigate the sample complexity of networks with bounds on the magnitude of its weights... To establish our results we develop a new technique to analyze the sample complexity of families H of predictors. We start by defining a new notion of a randomized approximate description of functions f : X Rd. We then show that if there is a way to approximately describe functions in a class H using d bits, then d ϵ2 examples suffices to guarantee uniform convergence. Namely, that the empirical loss of all the functions in the class is ϵ-close to the true loss. Finally, we develop a set of tools for calculating the approximate description length of classes of functions that can be presented as a composition of linear function classes and non-linear functions. |
| Researcher Affiliation | Collaboration | Amit Daniely Hebrew University and Google Research Tel-Aviv amit.daniely@mail.huji.ac.il Elad Granot Hebrew University elad.granot@mail.huji.ac.il |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any concrete access information (e.g., a specific repository link or an explicit statement of code release) for the methodology described. |
| Open Datasets | No | The paper is theoretical and does not conduct experiments involving datasets, thus no information about public dataset availability is present. |
| Dataset Splits | No | The paper is theoretical and does not describe experiments with data, so there is no mention of training/validation/test dataset splits. |
| Hardware Specification | No | The paper is theoretical and does not detail any computational experiments, thus no specific hardware details are provided. |
| Software Dependencies | No | The paper is theoretical and does not describe any software implementations or dependencies with version numbers. |
| Experiment Setup | No | The paper is theoretical and does not include details about an experimental setup, such as hyperparameter values or training configurations. |