Generalization Bounds using Data-Dependent Fractal Dimensions
Authors: Benjamin Dupuis, George Deligiannidis, Umut Simsekli
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We support our theory with experiments conducted on various settings. Finally, we illustrate our bounds on experiments using various neural networks. |
| Researcher Affiliation | Academia | Benjamin Dupuis 1 2 3 George Deligiannidis 4 5 Umut S ims ekli 1 2 3 6 1Inria 2Ecole Normale Sup erieure, Paris, France 3PSL Research University, Paris, France 4Department of Statistics, University of Oxford, Oxford, UK 5The Alan Turing Institute, London, UK 6CNRS. |
| Pseudocode | No | The paper describes procedural steps for computation and experiments but does not include any formal pseudocode or algorithm blocks. |
| Open Source Code | Yes | Python code for numerical experiments is available at https://github.com/benjiDupuis/ data_dependent_dimensions. |
| Open Datasets | Yes | (i) regression experiment with Fully Connected Networks of 5 (FCN-5) and 7 (FCN-7) layers trained on the California Housing Dataset (CHD) (Kelley Pace & Barry, 1997), (ii) training FCN-5 and FCN-7 networks on the MNIST dataset (Lecun et al., 1998) and (iii) training Alex Net (Krizhevsky et al., 2017) on the CIFAR-10 dataset (Krizhevsky et al., 2014). |
| Dataset Splits | Yes | We trained FCN-5 and FCN-7 of width 200 (for each inner layer) on a training set corresponding to a random subset of 80% of the 20640 points of the California Housing Dataset, using the remaining 20% for validation. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used, such as GPU or CPU models. It mentions 'computational resources' in Appendix D.3 but no specifications. |
| Software Dependencies | No | The paper mentions using 'Python code' and 'the PH software provided in (P erez et al., 2021)', but it does not specify version numbers for any software dependencies like Python, PyTorch, or specific libraries used. |
| Experiment Setup | Yes | We made both learning rate and batch size vary across a 6x6 grid. All hyperparameter configurations are available in Section C. For classification experiments: Learning rate vary in the set [5.10 3, 10 1] and batch size vary in [32, 256]. For regression experiments: Learning rate vary in the set [1.10 3, 10 2] and batch size vary in [32, 200]. |