Towards Fast Computation of Certified Robustness for ReLU Networks
Authors: Lily Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Luca Daniel, Duane Boning, Inderjit Dhillon
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that (1) our methods deliver bounds close to (the gap is 2-3X) exact minimum distortions found by Reluplex in small networks while our algorithms are more than 10,000 times faster; (2) our methods deliver similar quality of bounds (the gap is within 35% and usually around 10%; sometimes our bounds are even better) for larger networks compared to the methods based on solving linear programming problems but our algorithms are 3314,000 times faster; (3) our method is capable of solving large MNIST and CIFAR networks up to 7 layers with more than 10,000 neurons within tens of seconds on a single CPU core. In this section, we perform extensive experiments to evaluate the performance of our proposed two lower-bound based robustness certificates on networks with different sizes and with different defending techniques during training process. |
| Researcher Affiliation | Academia | 1Massachusetts Institute of Technology, Cambridge, MA 2UC Davis, Davis, CA 3Harvard University, Cambridge, MA 4UT Austin, Austin, TX. |
| Pseudocode | Yes | We list our complete algorithm, Fast-Lin, in Appendix D. We list our full procedure, Fast-Lip, in Appendix D. |
| Open Source Code | Yes | https://github.com/huanzhang12/Certified Re LURobustness |
| Open Datasets | Yes | our method is capable of solving large MNIST and CIFAR networks up to 7 layers with more than 10,000 neurons within tens of seconds on a single CPU core. |
| Dataset Splits | No | The paper mentions using "100 random test images" but does not specify training, validation, or test split percentages or detailed methodology for data partitioning. |
| Hardware Specification | No | running time (per image) for all methods is measured on a single CPU core. While 'single CPU core' is a hardware detail, it lacks specific model numbers or types. |
| Software Dependencies | No | The paper does not explicitly list specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions, or solver versions) needed to reproduce the experiments. |
| Experiment Setup | Yes | Table 2. Comparison of the lower bounds for distortion found by our algorithms on models with defensive distillation (DD) (Papernot et al., 2016) with temperature = 100 and adversarial training (Madry et al., 2018) with = 0.3 for three targeted attack classes. |