Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics
Authors: Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli, Daniel LK Yamins, Hidenori Tanaka
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically validate our analytic expressions for learning dynamics on VGG-16 trained on Tiny Image Net. |
| Researcher Affiliation | Collaboration | Stanford University Physics & Informatics Laboratories, NTT Research, Inc. |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | An open source version of our code, used to generate all the figures in this paper, is available at github.com/danielkunin/neural-mechanics. |
| Open Datasets | Yes | Dataset. While we ran some initial experiments on Cifar-100, the dataset used in all the empirical figures in this documents was Tiny Imagenet. |
| Dataset Splits | No | The paper mentions 'Tiny Image Net' and training parameters like '100 epochs' and 'batch size S = 256', but does not specify explicit training/validation/test split percentages or sample counts, nor does it refer to predefined splits with citations. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used for experiments, such as specific GPU/CPU models or detailed computer specifications. |
| Software Dependencies | No | The paper mentions 'Py Torch' as a tool but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | All models were initialized using Kaiming Normal, and no learning rate drops or warmup were used. Model Dataset Epochs Batch size Opt. LR Mom. WD Damp. VGG-16 Tiny Image Net 100 256 SGD [0.1, 0.01] [0, 0.001, 0.0005, 0.0001] 0 VGG-16 w/BN Tiny Image Net 100 256 SGD [0.1, 0.01] [0, 0.001, 0.0005, 0.0001] 0 VGG-16 Tiny Image Net 100 128 SGDM 0.1 [0, 0.9, 0.99] [0, 0.001, 0.0005, 0.0001] 0 VGG-16 w/BN Tiny Image Net 100 128 SGDM 0.1 [0, 0.9, 0.99] [0, 0.001, 0.0005, 0.0001] 0 |