Stochastic Activation Pruning for Robust Adversarial Defense
Authors: Guneet S. Dhillon, Kamyar Azizzadenesheli, Zachary C. Lipton, Jeremy D. Bernstein, Jean Kossaifi, Aran Khanna, Animashree Anandkumar
ICLR 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments to evaluate SAP address two tasks: image classification and reinforcement learning. We apply the method to the Re LU activation maps at each layer of the pretrained neural networks. To create adversarial examples in our evaluation, we use FGSM, x = λ sign(J (Mp(θ), x, y)). |
| Researcher Affiliation | Collaboration | Guneet S. Dhillon1,2, Kamyar Azizzadenesheli3, Zachary C. Lipton1,4, Jeremy Bernstein1,5, Jean Kossaifi1,6, Aran Khanna1, Anima Anandkumar1,5 1Amazon AI, 2UT Austin, 3UC Irvine, 4CMU, 5Caltech, 6Imperial College London |
| Pseudocode | Yes | Algorithm 1 Stochastic Activation Pruning (SAP) |
| Open Source Code | Yes | All the implementations were coded in MXNet framework (Chen et al., 2015) and sample code is available at https://github.com/Guneet-Dhillon/Stochastic-Activation-Pruning |
| Open Datasets | Yes | The CIFAR-10 dataset (Krizhevsky & Hinton, 2009) was used for the image classification domain. |
| Dataset Splits | Yes | It was trained on a dataset consisting of 80% un-perturbed data and 20% adversarially perturbed data, generated on the model from the previous epoch, with λ = 2. This achieved an accuracy of 75.0% on the un-perturbed validation set. |
| Hardware Specification | No | The paper states, 'Computing a single backward pass of the SAP-100 model for 512 examples takes 20 seconds on 8 GPUs.' While GPUs are mentioned, no specific model numbers, memory details, or other hardware specifications are provided. |
| Software Dependencies | No | The paper mentions 'MXNet framework (Chen et al., 2015)' but does not provide a specific version number for this framework or any other software dependencies. |
| Experiment Setup | Yes | We trained a Res Net-20 model (He et al., 2016) using SGD, with minibatches of size 512, momentum of 0.9, weight decay of 0.0001, and a learning rate of 0.5 for the first 100 epochs, then 0.05 for the next 30 epochs, and then 0.005 for the next 20 epochs. |