Adversarial Robustness Against the Union of Multiple Perturbation Models
Authors: Pratyush Maini, Eric Wong, Zico Kolter
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | With this approach, we are able to train standard architectures which are simultaneously robust against ℓ , ℓ2, and ℓ1 attacks, outperforming past approaches on the MNIST and CIFAR10 datasets and achieving adversarial accuracy of 47.0% against the union of (ℓ , ℓ2, ℓ1) perturbations with radius = (0.03, 0.5, 12) on the latter, improving upon previous approaches which achieve 40.6% accuracy. |
| Researcher Affiliation | Collaboration | Pratyush Maini 1 Eric Wong 2 J. Zico Kolter 3 4 1Department of Computer Science and Engineering, IIT Delhi, India 2Machine Learning Department, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA 3Computer Science Department, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA 4Bosch Center for Artificial Intelligence, Pittsburgh, Pennsylvania, USA. |
| Pseudocode | Yes | Algorithm 1 Multi steepest descent for learning classifiers that are simultaneously robust to ℓp attacks for p S |
| Open Source Code | Yes | Code for reproducing all the results can be found at: https://github.com/locuslab/robust union. |
| Open Datasets | Yes | We train models using MSD, MAX and AVG approaches for both MNIST and CIFAR10 datasets. |
| Dataset Splits | No | All attacks were run on a subset of the first 1000 test examples with 10 random restarts... The paper refers to MNIST and CIFAR10 datasets which have standard splits, but it does not explicitly state the training/validation/test split percentages or sample counts for these datasets, nor does it refer to a citation for the split. It only explicitly mentions the test subset size used for evaluation. |
| Hardware Specification | Yes | All experiments were run on modest amounts of GPU hardware (e.g. a single 1080ti). |
| Software Dependencies | No | We perform an extensive evaluation of these models with a broad suite of both gradient and non-gradient based attacks using Foolbox2 (the same attacks used by Schott et al. (2019)). The footnote 2 links to https://github.com/bethgelab/foolbox (Rauber et al., 2017) but no specific version is indicated for Foolbox itself. |
| Experiment Setup | Yes | A complete description of the hyperparameters used is in Appendix C. All reported ǫ are for images scaled to be between the range [0, 1]. All experiments were run on modest amounts of GPU hardware (e.g. a single 1080ti). We make 10 random restarts for each of the results mentioned hereon for both MNIST and CIFAR10 3. |