Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
Authors: Kai Y. Xiao, Vincent Tjeng, Nur Muhammad (Mahi) Shafiullah, Aleksander Madry
ICLR 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our techniques enable us to train weight-sparse and Re LU stable networks for MNIST and CIFAR10 that can be verified significantly faster. Specifically, by combining natural methods for inducing weight sparsity with a robust adversarial training procedure (cf. Goodfellow et al. (2015)), we are able to train networks for which almost 90% of inputs can be verified in an amount of time that is small2 compared to previous verification techniques. Then, by also adding our regularization technique for inducing Re LU stability, we are able to train models that can be verified an additional 4 13x times as fast while maintaining state-of-the-art accuracy on MNIST. Our techniques show similar improvements for exact verification of CIFAR models. In particular, we achieve the following verification speed and provable robustness results for ℓ norm-bound adversaries: Dataset Epsilon Provable Adversarial Accuracy Average Solve Time (s) MNIST ϵ = 0.1 94.33% 0.49 ϵ = 0.2 89.79% 1.13 ϵ = 0.3 80.68% 2.78 CIFAR ϵ = 2/255 45.93% 13.50 ϵ = 8/255 20.27% 22.33 |
| Researcher Affiliation | Academia | Kai Y. Xiao Vincent Tjeng Nur Muhammad (Mahi) Shafiullah Aleksander M adry Massachusetts Institute of Technology Cambridge, MA 02139 {kaix, vtjeng, nshafiul, madry}@mit.edu |
| Pseudocode | No | The paper describes methods like naive interval arithmetic and improved interval arithmetic in text, but does not include formally structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/Madry Lab/relu_stable. |
| Open Datasets | Yes | Our techniques enable us to train weight-sparse and Re LU stable networks for MNIST and CIFAR10 that can be verified significantly faster. |
| Dataset Splits | No | The paper mentions training on MNIST and CIFAR datasets and evaluating on a 'test set', but does not explicitly provide details about training/validation/test dataset splits (percentages or specific counts for validation). |
| Hardware Specification | Yes | Model solves were parallelized over 8 CPU cores with Intel Xeon CPUs @ 2.20GHz processors. We used computers with 8 32GB of RAM, depending on the size of the model being verified. |
| Software Dependencies | Yes | The solver we chose was the commercial Gurobi Solver, which uses a branch-and-bound method for solving MILPs. The paper later states: 'We ran our experiments using the commercial Gurobi Solver (version 7.5.2)'. |
| Experiment Setup | Yes | For MNIST, we trained for 70 epochs using the Adam optimizer (Kingma and Ba, 2015) with a learning rate of 1e 4 and a batch size of 32. For CIFAR, we trained for 250 epochs using the Adam optimizer with a learning rate of 1e 4. When using naive IA, we used a batch size of 128, and when using improved IA, we used a batch size of 16. We also provide a table with weights chosen using line search for ℓ1 regularization and RS Loss in each setting (Table 6). |