Perceptual Adversarial Robustness: Defense Against Unseen Threat Models
Authors: Cassidy Laidlaw, Sahil Singla, Soheil Feizi
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We test PAT on CIFAR-10 and Image Net-100 against five diverse adversarial attacks: L2, L , spatial, recoloring, and JPEG. We find that PAT achieves state-of-the-art robustness against the union of these five attacks more than doubling the accuracy over the next best model without training against any of them. We compare Perceptual Adversarial Training (PAT) to adversarial training against narrow threat models (Lp, spatial, etc.) on CIFAR-10 (Krizhevsky and Hinton, 2009) and Image Net-100 (the subset of Image Net (Russakovsky et al., 2015) containing every tenth class by Word Net ID order). |
| Researcher Affiliation | Academia | Cassidy Laidlaw University of Maryland claidlaw@umd.edu Sahil Singla University of Maryland ssingla@cs.umd.edu Soheil Feizi University of Maryland sfeizi@cs.umd.edu |
| Pseudocode | Yes | Algorithm 1 Perceptual PGD (PPGD) [...] Algorithm 5 Perceptual Projection (Bisection Method) |
| Open Source Code | Yes | Code and data can be downloaded at https://github.com/cassidylaidlaw/perceptual-advex. |
| Open Datasets | Yes | We test PAT on CIFAR-10 (Krizhevsky and Hinton, 2009) and Image Net-100 (the subset of Image Net (Russakovsky et al., 2015) containing every tenth class by Word Net ID order). The CIFAR-10 dataset can be obtained from https://www.cs.toronto.edu/~kriz/cifar. html. The Image Net-100 dataset is a subset of the Image Net Large Scale Visual Recognition Challenge (2012) (Russakovsky et al., 2015) including only every tenth class by Word Net ID order. It can be obtained from http://www.image-net.org/download-images. |
| Dataset Splits | Yes | We test PAT on CIFAR-10 (Krizhevsky and Hinton, 2009) and Image Net-100 (the subset of Image Net (Russakovsky et al., 2015) containing every tenth class by Word Net ID order). The CIFAR-10 dataset can be obtained from https://www.cs.toronto.edu/~kriz/cifar. html. The Image Net-100 dataset is a subset of the Image Net Large Scale Visual Recognition Challenge (2012) (Russakovsky et al., 2015) including only every tenth class by Word Net ID order. |
| Hardware Specification | Yes | Self-bounded PAT takes about 12 hours to train for CIFAR-10 on an Nvidia RTX 2080 Ti GPU, and about 5 days to train for Image Net-100 on 4 GPUs. |
| Software Dependencies | No | The paper mentions implementing the methods using 'Py Torch (Paszke et al., 2017)' but does not specify a version number for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | Table 12: Hyperparameters for the adversarial training experiments on CIFAR-10 and Image Net-100. For CIFAR-10, hyperparameters are similar to those used by Zhang et al. (2019a). For Image Net-100, hyperparameters are similar to those used by Kang et al. (2019). Parameter CIFAR-10 Image Net-100 Architecture Res Net-50 Res Net-50 Number of parameters 23,520,842 23,712,932 Optimizer SGD SGD Momentum 0.9 0.9 Weight decay 2 10 4 10 4 Batch size 50 128 Training epochs 100 90 Initial learning rate 0.1 0.1 Learning rate drop epochs ( 0.1 drop) 75, 90 30, 60, 80 Attack iterations (train) 10 10 Attack iterations (test) 200 200 |