Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Prior Convictions: Black-box Adversarial Attacks with Bandits and Priors
Authors: Andrew Ilyas, Logan Engstrom, Aleksander Madry
ICLR 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our approach on the task of generating black-box adversarial examples, where the methods obtained from integrating two example priors significantly outperform state-of-the-art approaches. We evaluate our bandit approach described in Section 3 and the natural evolutionary strategies (NES) approach of Ilyas et al. (2017) on their effectiveness in generating untargeted adversarial examples. |
| Researcher Affiliation | Academia | Andrew Ilyas , Logan Engstrom , Aleksander M adry EMAIL MIT CSAIL |
| Pseudocode | Yes | Algorithm 1 Gradient Estimation with Bandit Optimization, Algorithm 2 Single-query spherical estimate of v L(x, y), v, Algorithm 3 Adversarial Example Generation with Bandit Optimization for ℓ2 norm perturbations |
| Open Source Code | Yes | 1The code for reproducing our work is available at https://git.io/fAjOJ. |
| Open Datasets | Yes | We consider both the ℓ2 and ℓ threat models on the Image Net (Russakovsky et al., 2015) dataset, in terms of success rate and query complexity. Finally, we also have similar results for CIFAR-10 under the ℓ threat model, which can be found in Appendix E. |
| Dataset Splits | Yes | We use 10,000 and 1,000 randomly selected images (scaled to [0, 1]) to evaluate all approaches on Image Net and CIFAR-10 respectively. gradients of 5,000 randomly chosen example images in the Image Net validation set. |
| Hardware Specification | No | No specific hardware details (like GPU models, CPU types, or memory) used for running experiments were mentioned. |
| Software Dependencies | No | The paper mentions 'PyTorch Image Net classifiers' but does not provide specific version numbers for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | Table 2: Hyperparameters for the NES approach. Table 3: Hyperparameters for the bandits approach (variables names as used in pseudocode). |