Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
Authors: Francesco Croce, Matthias Hein
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We run experiments on MNIST, CIFAR-10 (Krizhevsky et al., 2014) and Restricted Image Net (Tsipras et al., 2019). For each dataset we consider a normally trained model (plain) and two adversarially trained ones as in (Madry et al., 2018) wrt the l -norm (l -AT) and the l2-norm (l2-AT) (see supplementary material for details). [...] We compare the performance of FAB-attack1 to those of attacks representing the state-of-the-art in each norm [...]. We report the complete results in the supplementary material, while we summarize them in Table 1 (MNIST and CIFAR-10 aggregated, as we used the same attacks) and Table 2 (Restricted Image Net). Our FAB-attack achieves the best results in all statistics for every norm (with the only exception of "max diff. to best" in l ) on MNIST+CIFAR-10. |
| Researcher Affiliation | Academia | 1University of Tübingen, Germany. Correspondence to: F. Croce <francesco.croce@uni-tuebingen.de>. |
| Pseudocode | Yes | Algorithm 1 FAB-attack |
| Open Source Code | Yes | 1https://github.com/fra31/fab-attack |
| Open Datasets | Yes | We run experiments on MNIST, CIFAR-10 (Krizhevsky et al., 2014) and Restricted Image Net (Tsipras et al., 2019). |
| Dataset Splits | Yes | In total we have 5 thresholds × 6 models = 30 cases for each of the 3 norms. *Note that for FAB-10 (i.e. with 10 restarts) the "# best" is computed excluding the results of FAB-100. (on the frst 1000 points for l and l1, 500 for l2, of the test sets)." (Table 1 caption) and "on the frst 500 points of the validation set" (Table 2 caption). |
| Hardware Specification | No | No specific hardware details (like exact GPU/CPU models, processor types, or memory amounts) used for experiments were provided. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., library or solver names with version numbers like Python 3.8, PyTorch 1.9) were provided. |
| Experiment Setup | Yes | For FAB-attack we use always β = 0.9 and on MNIST and CIFAR-10: αmax = 0.1, η = 1.05 and on Restricted Image Net: αmax = 0.05, η = 1.3. These parameters are the same for all norms. ... As a result we use for PGD wrt l step size ϵ/10 and the direction is the sign of the cross entropy loss, for PGD wrt l2 we do a step in the direction of the l2-normalized gradient with step size ϵ/4, for PGD wrt l1 we use the gradient step suggested in (Tramèr & Boneh, 2019) (with sparsity levels of 1% for MNIST and 10% for CIFAR-10 and Restricted Image Net) with step size ϵ/2. |