The Crucial Role of Normalization in Sharpness-Aware Minimization
Authors: Yan Dai, Kwangjun Ahn, Suvrit Sra
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our conclusions are backed by various experiments. |
| Researcher Affiliation | Academia | Yan Dai IIIS, Tsinghua University yan-dai20@mails.tsinghua.edu.cn Kwangjun Ahn EECS, MIT kjahn@mit.edu Suvrit Sra TU Munich / MIT suvrit@mit.edu |
| Pseudocode | No | The paper contains mathematical equations and derivations but no structured pseudocode or algorithm blocks are explicitly presented or labeled. |
| Open Source Code | No | The paper does not contain an explicit statement indicating the release of source code for the described methodology, nor does it provide a direct link to a code repository. |
| Open Datasets | Yes | We train a ResNet18 on the CIFAR-10 dataset |
| Dataset Splits | No | The paper mentions training on CIFAR-10 and using specific initializations/checkpoints, but it does not explicitly state the dataset splits (e.g., percentages or counts for training, validation, or testing sets) used for reproduction. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used for its experiments, such as specific GPU or CPU models, or details about the computing environment. |
| Software Dependencies | No | The paper mentions deep learning models and optimizers but does not provide specific software dependency details, such as library names with version numbers (e.g., 'PyTorch 1.9', 'Python 3.8') used for the experiments. |
| Experiment Setup | Yes | Figure 5: Training ResNet18 on CIFAR-10 from a bad global minimum (η = 0.001, batch size = 128). |