Adaptive Randomized Smoothing: Certified Adversarial Robustness for Multi-Step Defences
Authors: Saiyue Lyu, Shadab Shaikh, Frederick Shpilevskiy, Evan Shelhamer, Mathias Lécuyer
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We instantiate ARS on deep image classification to certify predictions against adversarial examples of bounded L norm. In the L threat model, ARS enables flexible adaptation through high-dimensional inputdependent masking. We design adaptivity benchmarks, based on CIFAR-10 and Celeb A, and show that ARS improves standard test accuracy by 1 to 15% points. On Image Net, ARS improves certified test accuracy by up to 1.6% points over standard RS without adaptivity. |
| Researcher Affiliation | Collaboration | 1University of British Columbia, 2Google Deep Mind |
| Pseudocode | No | The paper describes the two-step ARS process and architecture in text and Figure 1, but it does not include a formally structured "Pseudocode" or "Algorithm" block. |
| Open Source Code | Yes | Our code is available at https: //github.com/ubc-systopia/adaptive-randomized-smoothing. |
| Open Datasets | Yes | We evaluate on CIFAR-10 (Krizhevsky, 2009) in 4.1, Celeb A (Liu et al., 2015) (specifically the unaligned HD-Celeb A-Cropper edition) in 4.2, and Image Net (Deng et al., 2009) in 4.3. |
| Dataset Splits | No | The paper mentions "train and test sets" for the background images and discusses evaluation on subsets of the test set, but it does not explicitly specify the training/validation/test dataset splits (e.g., percentages or exact counts) for the main datasets (CIFAR-10, Celeb A, ImageNet) or how validation sets were used. |
| Hardware Specification | Yes | To certify a single input (k = 32), Cohen et al. (2019) takes 12 seconds while ARS takes 26 seconds (as measured on an NVIDIA A100 80Gb GPU). |
| Software Dependencies | No | The paper does not explicitly list specific software dependencies with version numbers (e.g., Python, PyTorch, CUDA versions) required to replicate the experiments. |
| Experiment Setup | Yes | We set the failure probability of the certification procedure to 0.05, use 100 samples to select the most probable class, and 50,000 samples for the Monte Carlo estimate of p+. For ARS, our mask model w is a simplified U-Net (Ronneberger et al., 2015) (see Appendix C.1 for details). For the noise budget, we find that a fixed budget split performs reliably, and so in all experiments we split by σ1 = σ2 = . Table 4: Hyperparameters for training ARS. Check Appendix C.3 for more details of CIFAR-10 hyperparameters. |