Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
How Many Perturbations Break This Model? Evaluating Robustness Beyond Adversarial Accuracy
Authors: Raphael Olivier, Bhiksha Raj
ICML 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We run experiments in L2 and L perturbations on the CIFAR10 dataset (Krizhevsky et al., 2009). Additional results on Image Net can be found in Appendix C. We use attack radii ϵ = 0.5 and ϵ = 8/255 respectively. The results of adversarially defended models are reported in Table 1. |
| Researcher Affiliation | Academia | Raphael Olivier 1 Bhiksha Raj 1 1Language Technologies Institute, Carnegie Mellon University, Pittsburgh, USA. Correspondence to: Raphael Olivier <EMAIL>. |
| Pseudocode | Yes | Algorithm 1 L2 sparsity computation algorithm |
| Open Source Code | Yes | Our code is available at https://github.com/ Raphael Olivier/sparsity |
| Open Datasets | Yes | We run experiments in L2 and L perturbations on the CIFAR10 dataset (Krizhevsky et al., 2009). |
| Dataset Splits | No | We report averaged values of sparsity over the first 1000 vulnerable inputs in the CIFAR10 test set and, for each input, 100 random directions. |
| Hardware Specification | Yes | On CIFAR10, computing adversarial sparsity around an input point with 100 directions, 10 search steps, and 20 PGD iterations takes a few seconds for a Res Net-18 model on an Nvidia RTX 2080 Ti. |
| Software Dependencies | No | The paper mentions using specific attacks (PGD, APGD-CE, APGD-DLR) and models (Res Net-18), but does not provide specific software environment versions like Python, PyTorch/TensorFlow versions, or library versions. |
| Experiment Setup | Yes | We run experiments in L2 and L perturbations on the CIFAR10 dataset (Krizhevsky et al., 2009). Additional results on Image Net can be found in Appendix C. We use attack radii ϵ = 0.5 and ϵ = 8/255 respectively. |