Automatic Data Augmentation via Invariance-Constrained Learning
Authors: Ignacio Hounie, Luiz F. O. Chamon, Alejandro Ribeiro
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate empirically our theoretical developments in automatic data augmentation benchmarks for CIFAR and Image Net100 datasets. Furthermore, our experiments show how this approach can be used to gather insights on the actual symmetries underlying a learning task. |
| Researcher Affiliation | Academia | 1University of Pennsylvania 2University of Stuttgart. Correspondence to: Ignacio Hounie <ihounie@seas.upenn.edu>. |
| Pseudocode | Yes | Algorithm 1 describes the implementation of independent MH with a uniform proposal. By keeping only one sample (m = 1) we recover the usual augmentation setting, that yields one augmentation per sample in the training batch. Algorithm 2 Primal-Dual Augmentation |
| Open Source Code | No | The paper does not contain an explicit statement or link indicating that the authors are releasing their source code. |
| Open Datasets | Yes | We validate empirically our theoretical developments in automatic data augmentation benchmarks for CIFAR and Image Net100 datasets. We showcase this on datasets with artificial invariances, following the setup of (Immer et al., 2022). Namely, we apply rotations, translations or scalings, independently drawn from the uniform distributions, to each sample in the MNIST (Le Cun et al., 2010) and Fashion MNIST (Xiao et al., 2017) datasets. |
| Dataset Splits | Yes | The constraint level was determined by a grid search targeting cross-validation accuracy. |
| Hardware Specification | Yes | Table 11. Time per epoch for Wide Resnet 40-2 in CIFAR 10 dataset, on a workstation with one NVIDIA RTX 3090 GPU and AMD Threadripper 3960X (24 cores, 3.80 GHz) CPU. |
| Software Dependencies | No | The paper mentions software like 'Pillow implementation', 'SGD with Nesterov Momentum', and 'Adam (Kingma and Ba, 2014)'. However, it does not provide specific version numbers for these software dependencies, which are necessary for reproducible description. |
| Experiment Setup | Yes | Except for epoch ablation and Imagenet-100 experiments, we use SGD with Nesterov Momentum and a learning rate of 0.1, a batch size of 128, a 5e-4 weight decay. In Imagenet100 experiments, we use Adam (Kingma and Ba, 2014) and a learning rate of 5e-4. We use a cosine learning rate decay schedule and train for 270 epochs for tiny Imagenet and 200 epochs for all other datasets. |