The Hessian perspective into the Nature of Convolutional Neural Networks
Authors: Sidak Pal Singh, Thomas Hofmann, Bernhard Schölkopf
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically validate our upper bounds in a variety of settings, in particular, with both linear and Re LU activations, MSE and CE loss, as well as on datasets such as CIFAR10, Fashion MNIST, MNIST, and a synthetic dataset. |
| Researcher Affiliation | Academia | 1ETH Zürich, Switzerland 2MPI for Intelligent Systems, Tübingen, Germany. |
| Pseudocode | No | The paper describes mathematical proofs, theorems, and derivations but does not include any pseudocode or algorithm blocks. |
| Open Source Code | Yes | The corresponding code can be found under https://github.com/sidak/hessian_perspective_cnns. |
| Open Datasets | Yes | We empirically validate our upper bounds in a variety of settings, in particular, with both linear and Re LU activations, MSE and CE loss, as well as on datasets such as CIFAR10, Fashion MNIST, MNIST, and a synthetic dataset. |
| Dataset Splits | No | The paper mentions using datasets like CIFAR10, but it does not specify explicit training, validation, or test splits (e.g., 80/10/10 split percentages or sample counts for each set) needed for reproducibility. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., CPU, GPU models, memory) used to run the experiments. It only mentions general experimental settings. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies (e.g., Python, PyTorch, TensorFlow, specific solvers or libraries) used in their experiments, which is necessary for reproducibility. |
| Experiment Setup | Yes | We empirically validate our upper bounds in a variety of settings, in particular, with both linear and Re LU activations, MSE and CE loss, as well as on datasets such as CIFAR10, Fashion MNIST, MNIST, and a synthetic dataset. Following (Singh et al., 2021), to rigorously illustrate the match with our bounds, we compute the exact Hessians, without approximations, and in Float64 precision. |