Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond
Authors: Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, Cho-Jui Hsieh
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Table 2: Error rates of different certifiably trained models on CIFAR-10 and Tiny-Image Net datasets (results on downscaled Image Net are in Table 4). Table 3: Per-epoch training time and memory usage of 4 large models on CIFAR-10 with batch size 256, and 3 large models on Tiny-Image Net with batch size 100. |
| Researcher Affiliation | Academia | 1Northeastern University 2Tsinghua University 3UCLA 4DCST, THUAI, SKLits, BNRist, Tsinghua University 5Lawrence Livermore National Laboratory |
| Pseudocode | Yes | Algorithm 1 Forward Mode Bound Propagation on General Computational Graphs, Algorithm 2 Backward Mode Bound Propagation on a General Computational Graph |
| Open Source Code | Yes | Our open source library is available at https://github.com/Kaidi Xu/auto_Li RPA. |
| Open Datasets | Yes | We report results on CIFAR-10 [25] with ℓ perturbation ϵ=8/255 and Tiny-Image Net with ϵ=1/255 in Table 2, and Downscaled-Image Net [5] which has 1, 000 class labels with ℓ perturbation ϵ=1/255 in Table 4. |
| Dataset Splits | No | The paper discusses training and test errors but does not explicitly provide details about validation dataset splits or how they were derived for reproducibility. |
| Hardware Specification | Yes | We use the same batch size 256 for all settings and conduct the experiments on 4 Nvidia GTX 1080Ti GPUs. |
| Software Dependencies | No | The paper mentions using “Py Torch” but does not specify version numbers for any software dependencies needed to replicate the experiments. |
| Experiment Setup | Yes | We use the same batch size 256 for all settings and conduct the experiments on 4 Nvidia GTX 1080Ti GPUs. We provide detailed hyperparameters in Appendix C.1. We consider synonym-based word substitution with δ 6 (up to 6 word substitutions). We provide more backgrounds and training details in Appendix C.2. |