Equi-normalization of Neural Networks
Authors: Pierre Stock, Benjamin Graham, Rémi Gribonval, Hervé Jégou
ICLR 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We analyze our approach by carrying out experiments on the MNIST and CIFAR-10 datasets and on the more challenging Image Net dataset. |
| Researcher Affiliation | Collaboration | Pierre Stock1,2, Benjamin Graham1, R emi Gribonval2 and Herv e J egou1 1Facebook AI Research 2Univ Rennes, Inria, CNRS, IRISA |
| Pseudocode | Yes | Algorithm 1: Pseudo-code of Equi-normalization |
| Open Source Code | Yes | Our code is available at https://github.com/facebookresearch/enorm. |
| Open Datasets | Yes | We analyze our approach by carrying out experiments on the MNIST and CIFAR-10 datasets and on the more challenging Image Net dataset. |
| Dataset Splits | Yes | We split the train set into one training set (40,000 images) and one validation set (10,000 images). |
| Hardware Specification | No | The paper mentions 'We train on 8 GPUs' but does not specify the model or type of the GPUs (e.g., NVIDIA A100, Tesla V100), nor does it provide details about CPUs or memory. |
| Software Dependencies | Yes | Although the additional overhead of implicit ENorm is theoretically negligible, we observed an increase of the training time of a Res Net-18 by roughly 30% using Py Torch 4.0 (Paszke et al., 2017). |
| Experiment Setup | Yes | We select the learning rate in {0.001, 0.01, 0.1} and decay it linearly to zero. We use a batch size of 256 and SGD with no momentum and a weight decay of 0.001. |