Defending Against Image Corruptions Through Adversarial Augmentations
Authors: Dan Andrei Calian, Florian Stimberg, Olivia Wiles, Sylvestre-Alvise Rebuffi, András György, Timothy A Mann, Sven Gowal
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section we compare the performance of classifiers trained using our method (Ad A) and competing state-of-the-art methods (Aug Mix of Hendrycks et al., 2020b, Deep Augment of Hendrycks et al., 2020a) on (1) robustness to common image corruptions (on CIFAR-10-C & IMAGENET-C); (2) robustness to ℓp-norm bounded adversarial perturbations; and (3) generalization to distribution shifts on other variants of IMAGENET and CIFAR-10. |
| Researcher Affiliation | Industry | Dan A. Calian1 dancalian@deepmind.com Florian Stimberg1 stimberg@deepmind.com Olivia Wiles1 oawiles@deepmind.com Sylvestre-Alvise Rebuffi1 sylvestre@deepmind.com Andr as Gy orgy1 agyorgy@deepmind.com Timothy Mann2 mann.timothy@acm.org Sven Gowal1 sgowal@deepmind.com 1Deep Mind, 2Meta |
| Pseudocode | Yes | Algorithm 1 Ad A, our proposed method. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described in this paper, nor does it explicitly state that the code is open-source. |
| Open Datasets | Yes | On CIFAR-10 we train pre-activation Res Net50 (He et al., 2016b) models (as in Wong et al. (2020)) on the clean training set of CIFAR-10 (and evaluate on CIFAR-10-C and CIFAR-10.1); our models employ 3x3 kernels for the first convolutional layer, as in previous work (Hendrycks et al., 2020b). For IMAGENET we train standard Res Net50 classifiers on the training set of IMAGENET with standard data augmentation but 128x128 re-scaled image crops (due to the increased computational requirements of adversarial training) and evaluate on IMAGENET{C,R,v2}. |
| Dataset Splits | No | The paper discusses training and testing datasets but does not explicitly provide specific details about a validation dataset split or percentages used for hyperparameter tuning. |
| Hardware Specification | No | The paper mentions running experiments on 'GPU or TPU' but does not specify exact models (e.g., NVIDIA A100, Tesla V100, TPU v3) or other detailed hardware specifications. |
| Software Dependencies | No | The paper mentions using specific optimizers and learning rate schedules, but it does not provide specific software dependencies with version numbers (e.g., Python 3.x, PyTorch 1.x, CUDA 11.x). |
| Experiment Setup | Yes | For CIFAR-10 we train for 300 epochs with a batch size of 1024 and use a global weight decay of 10-4. For IMAGENET we train for 90 epochs with a batch size of 4096 and use a global weight decay of 5x10-4. We use a cosine learning rate schedule (Loshchilov & Hutter, 2017), without restarts, with 5 warm-up epochs, with an initial learning rate of 0.1 which is decayed to 0 at the end of training. We scale all learning rates using the linear scaling rule of Goyal et al. (2017), i.e., effective LR = max(LR batch size/256, LR). In Algorithm 1 the effective learning rate of the outer optimization is denoted by ηf. ... We use a step size equal to 1/4 of the median perturbation radius over all parameter blocks (for each backbone network individually). In Algorithm 1 this step size is denoted by ηc. For CIFAR-10 we use 10 steps; for IMAGENET we use only 3 steps (due to the increased computational requirements) but we increase the step size by a factor of 10/3 to compensate for the reduction in steps. |