Essentially No Barriers in Neural Network Energy Landscape
Authors: Felix Draxler, Kambis Veschgini, Manfred Salmhofer, Fred Hamprecht
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We connect minima of different CNNs, Res Nets (He et al., 2016) and Dense Nets (Huang et al., 2017) on the image classification tasks CIFAR10 and CIFAR100 (Krizhevsky & Hinton, 2009) using Auto NEB. |
| Researcher Affiliation | Academia | 1Heidelberg Collaboratory for Image Processing (HCI), IWR, Heidelberg University, D-69120 Heidelberg, Germany 2Institut für Theoretische Physik, Heidelberg University, D-69120 Heidelberg, Germany. |
| Pseudocode | Yes | Algorithm 1 NEB and Algorithm 2 Auto NEB |
| Open Source Code | Yes | Source code is available at https://github.com/ fdraxler/Py Torch-Auto NEB. |
| Open Datasets | Yes | We connect minima of different CNNs, Res Nets (He et al., 2016) and Dense Nets (Huang et al., 2017) on the image classification tasks CIFAR10 and CIFAR100 (Krizhevsky & Hinton, 2009) |
| Dataset Splits | No | The paper mentions training samples and batch sizes but does not explicitly provide information on validation dataset splits (percentages, counts, or methodology) needed for reproducibility. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments, such as GPU models, CPU specifications, or cloud computing instance types. |
| Software Dependencies | No | The paper mentions 'PyTorch-AutoNEB' in the GitHub link, implying the use of PyTorch, but does not provide specific version numbers for any software dependencies or libraries. |
| Experiment Setup | Yes | As optimiser, we use SGD with momentum 0.9 and l2regularisation with λ = 0.0001. The NEB cycles are configured with a learning rate decay: Four cycles of 1000 steps each with learning rate 0.1. Two cycles with 2000 steps and learning rate 0.1. Four cycles of 1000 steps with learning rate 0.01. No big improvement was seen in the last four cycles of 1000 steps each with a learning rate of 0.001. |