Ensemble Adversarial Training: Attacks and Defenses
Authors: Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel
ICLR 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On Image Net, Ensemble Adversarial Training yields models with strong robustness to black-box attacks. In particular, our most robust model won the first round of the NIPS 2017 competition on Defenses against Adversarial Attacks (Kurakin et al., 2017c). (Abstract) ... 4 EXPERIMENTS (Section title) |
| Researcher Affiliation | Collaboration | Florian Tram er Stanford University tramer@cs.stanford.edu Alexey Kurakin Google Brain kurakin@google.com Nicolas Papernot Pennsylvania State University ngp5056@cse.psu.edu Ian Goodfellow Google Brain goodfellow@google.com Dan Boneh Stanford University dabo@cs.stanford.edu Patrick Mc Daniel Pennsylvania State University mcdaniel@cse.psu.edu |
| Pseudocode | No | The paper describes methods like FGSM and R+FGSM using mathematical equations (e.g., Equation 2, Equation 7). However, it does not include any structured pseudocode blocks or algorithms in a traditional pseudocode format. |
| Open Source Code | Yes | 1We publicly released our model after the first round, and it could thereafter be targeted using white-box attacks. (Footnote 1, Page 2) |
| Open Datasets | Yes | On Image Net, Ensemble Adversarial Training yields models with strong robustness to black-box attacks. (Abstract) ... We train Inception v3 and Inception Res Net v2 models (Szegedy et al., 2016a) on Image Net, using the pre-trained models shown in Table 3. (Section 4.2) ... We re-iterate our Image Net experiments on MNIST. (Appendix C) |
| Dataset Splits | No | The paper mentions evaluating on "test inputs" and "test set" (e.g., "For 10,000 random test inputs, and ϵ = 16/256, we report error rates on white-box Step-LL..." in Table 4 caption). However, it does not explicitly state details about train/validation/test dataset splits, such as percentages or specific sample counts for each split, which are necessary for full reproducibility of data partitioning. |
| Hardware Specification | No | The paper mentions running experiments on "50 machines" for synchronous distributed training. However, it does not provide any specific hardware details such as GPU models, CPU types, or other detailed specifications of these machines. For example: "We use synchronous distributed training on 50 machines, with minibatches of size 16 (we did not pre-compute gradients, and thus lower the batch size to fit all models in memory)." (Section 4.2) |
| Software Dependencies | No | The paper mentions using "TensorFlow-Slim library" (Abadi et al., 2015) in Section 4.1, and the reference provides a URL for TensorFlow. However, it does not specify exact version numbers for TensorFlow itself or any other software libraries or dependencies used in the experiments, which is required for reproducibility. For example: "Mart ın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Man e, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Vi egas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. Tensor Flow: Large-scale machine learning on heterogeneous systems, 2015. URL https://www.tensorflow.org/. Software available from tensorflow.org." (References) |
| Experiment Setup | Yes | The paper provides specific details about the experimental setup and hyperparameters. For instance: "As in Kurakin et al. (2017b), we use RMSProp with a learning rate of 0.045, decayed by a factor of 0.94 every two epochs." (Section 4.2). It also describes the minibatch size and how examples are replaced: "Half of the examples in a minibatch are replaced by Step-LL examples." (Section 4.2). Other details like the ϵ value for attacks are mentioned in table captions (e.g., "We use Step LL with ϵ = 16/256..." in Table 1 caption). |