i-RevNet: Deep Invertible Networks
Authors: Jörn-Henrik Jacobsen, Arnold W.M. Smeulders, Edouard Oyallon
ICLR 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | i-Rev Nets achieve the same performance on Imagenet compared to similar non-invertible Rev Net and Res Net architectures (Gomez et al., 2017; He et al., 2016). To shed light on the mechanism underlying the generalization-ability of the learned representation, we show that i-Rev Nets progressively separate and contract signals with depth. Our results are evidence for an effective reduction of variability through a contraction with a recoverable input obtained from a series of one-to-one mappings. |
| Researcher Affiliation | Academia | University of Amsterdam joern.jacobsen@bethgelab.org Now at Bethgelab, University of T ubingen CVN, Centrale Sup elec, Universit e Paris-Saclay ; Galen team, INRIA Saclay Seque L team, INRIA Lille ; DI, ENS, Universit e PSL |
| Pseudocode | No | This leads to the following equations: xj+1 = Sj+1 xj xj+1 = xj + Fj+1 xj xj = S 1 j+1xj+1 xj = xj+1 Fj+1 xj (1) |
| Open Source Code | Yes | 1Code is available at: https://github.com/jhjacobsen/pytorch-i-revnet |
| Open Datasets | Yes | large-scale problems like Image Net. |
| Dataset Splits | Yes | We evaluate both classifiers for each model on the validation set of Image Net and report the Top-1 accuracy in Figure 6. |
| Hardware Specification | No | The dataset is processed for 600k iterations on a batch size of 256, distributed on 4GPUs. |
| Software Dependencies | No | 1Code is available at: https://github.com/jhjacobsen/pytorch-i-revnet |
| Experiment Setup | Yes | We train with SGD and momentum of 0.9. We regularized the model with a ℓ2 weight decay of 10 4 and batch normalization. The dataset is processed for 600k iterations on a batch size of 256, distributed on 4GPUs. The initial learning rate is 0.1, dropped by a factor of ten every 160k iterations. The images values are mapped to [0, 1] while following geometric transformations were applied: random scaling, random horizontal flipping, random cropping of size 2242, and finally color distortions. |