Information-theoretic analysis of generalization capability of learning algorithms
Authors: Aolin Xu, Maxim Raginsky
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We derive upper bounds on the generalization error of a learning algorithm in terms of the mutual information between its input and output. The bounds provide an information-theoretic understanding of generalization in learning problems, and give theoretical guidelines for striking the right balance between data fit and generalization by controlling the input-output mutual information. |
| Researcher Affiliation | Academia | Department of Electrical and Computer Engineering and Coordinated Science Laboratory, University of Illinois, Urbana, IL 61801, USA. |
| Pseudocode | No | The paper describes algorithms such as the Gibbs algorithm and noisy ERM, but does not provide any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not mention or provide access to any open-source code for the described methodology. |
| Open Datasets | No | The paper is theoretical and does not conduct experiments with specific datasets, thus no information on training data availability is provided. |
| Dataset Splits | No | The paper is theoretical and does not describe experimental validation, so no dataset split information (training, validation, test) is provided. |
| Hardware Specification | No | The paper is theoretical and does not describe any experimental setup or the hardware used for computations. |
| Software Dependencies | No | The paper is theoretical and does not describe any experimental setup or software dependencies with version numbers. |
| Experiment Setup | No | The paper is theoretical and does not detail any experimental setup, hyperparameters, or training configurations. |