(Certified!!) Adversarial Robustness for Free!
Authors: Nicholas Carlini, Florian Tramer, Krishnamurthy Dj Dvijotham, Leslie Rice, Mingjie Sun, J Zico Kolter
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate diffusion denoised smoothing on two standard datasets, CIFAR-10 and Image Net, and find it gives state-of-the-art certified ℓ2 robustness on both. |
| Researcher Affiliation | Collaboration | 1Google 2Carnegie Mellon University 3Bosch Center for AI |
| Pseudocode | Yes | Figure 1: Our approach can be implemented in under 15 lines of code, given an off-the-shelf classifier fclf and an off-the-shelf diffusion model denoise. The PREDICT function is adapted from Cohen et al. (2019) and takes as input a number of noise samples N and a statistical significance level η (0, 1) and inherits the same robustness certificate proved in Cohen et al. (2019). |
| Open Source Code | Yes | Code to reproduce our experiments is available at: https://github.com/ethz-privsec/diffusion_denoised_smoothing. |
| Open Datasets | Yes | We evaluate diffusion denoised smoothing on two standard datasets, CIFAR-10 and Image Net... Image Net: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248 255. Ieee, 2009. |
| Dataset Splits | Yes | Image Net configuration. We denoise Image Net images with the 552M-parameter classunconditional diffusion model from Dhariwal & Nichol (2021), and classify images with the 305Mparameter BEi T large model (Bao et al., 2022) which reaches a 88.6% top-1 validation accuracy using the implementation from timm (Wightman, 2019). |
| Hardware Specification | Yes | We obtain a throughput of 825 images per second through the diffusion model and Vi T classifier on an A100 GPU at a batch size of 1,000. |
| Software Dependencies | No | No specific version numbers for software dependencies were provided. The paper mentions: "We use the implementation from Hugging Face" and "using the implementation from timm (Wightman, 2019)". |
| Experiment Setup | Yes | On CIFAR-10, we draw N = 100,000 noise samples and on Image Net we draw N = 10,000 samples to certify the robustness following Cohen et al. (2019). ... As is standard in prior work, we perform randomized smoothing for three different noise magnitudes, σ {0.25, 0.5, 1.0}. |