Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence
Authors: Antoine Labatie, Dominic Masters, Zach Eaton-Rosen, Carlo Luschi
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our main contributions are as follows: ... (iii) we demonstrate on an extensive set of experiments that, by reproducing BN s beneļ¬cial properties, our batch-independent normalization approach consistently matches or exceeds BN s performance. |
| Researcher Affiliation | Industry | Antoine Labatie Dominic Masters Zach Eaton-Rosen Carlo Luschi Graphcore Research, UK |
| Pseudocode | No | The paper describes the operations for Proxy Normalization in mathematical equations (Eq. 7) and provides practical implementation details in Appendix B, but it does not present structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper states, 'The experimental results can be reproduced based on Appendices A, B.' but does not provide a direct link to a code repository or explicitly state that source code is being released or is available in supplementary materials. Appendices A and B provide detailed descriptions of the experimental setup and PN implementation, but no actual code files or links. |
| Open Datasets | Yes | While we focus on Image Net [75] in the main text of this paper, we report in Appendix A some additional results on CIFAR [76]. |
| Dataset Splits | Yes | All the experimental details are reported in Appendix A. ... ImageNet is a dataset of 1.28M training images and 50K validation images, used in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) of 2012. |
| Hardware Specification | Yes | Our experiments with batch-independent norms were run on Graphcore s MK1 and MK2 IPUs. |
| Software Dependencies | No | The paper states, 'All the experimental details are reported in Appendix A.' and refers to other works for training recipes, but it does not specify software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | All the experimental details are reported in Appendix A. ... We detail all our choices of regularization in Appendices A.1 and A.3.4. |