A New Family of Generalization Bounds Using Samplewise Evaluated CMI
Authors: Fredrik Hellström, Giuseppe Durisi
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We now consider three deep learning settings and evaluate the bounds derived in Section 2. To the best of our knowledge, the tightest average generalization bounds available in the literature for typical deep learning scenarios, such as CNNs trained on MNIST [41] or CIFAR10 [42], are found in [17]. These numerical results are based on [17, Eq. 22], which is a version of (4) where the e-CMI is replaced by the f-CMI. |
| Researcher Affiliation | Academia | Fredrik Hellström Chalmers University of Technology Gothenburg, Sweden frehells@chalmers.se Giuseppe Durisi Chalmers University of Technology Gothenburg, Sweden durisi@chalmers.se |
| Pseudocode | No | The paper provides theoretical results in the form of theorems and lemmas, but does not include any pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code for our experiments is largely based on the code from [17].3 Available: https://github.com/hrayrhar/f-CMI. (Footnote 3) |
| Open Datasets | Yes | First, we consider a CNN trained with Adam (a variant of SGD) on a binarized version of MNIST, where we only consider the digits 4 and 9. Next, in Figure 2b, we look at Res Net-50 pretrained on Image Net and fine-tuned using SGD on CIFAR10. |
| Dataset Splits | Yes | Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] This is specified in the supplementary material. |
| Hardware Specification | Yes | Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] This is specified in the supplementary material. |
| Software Dependencies | No | The paper mentions training algorithms like Adam and SGD but does not specify software versions for libraries or environments used for implementation (e.g., PyTorch version, Python version). |
| Experiment Setup | Yes | Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] This is specified in the supplementary material. |