Incremental Randomized Smoothing Certification
Authors: Shubham Ugare, Tarun Suresh, Debangshu Banerjee, Gagandeep Singh, Sasa Misailovic
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally demonstrate the effectiveness of our approach, showing up to 4.1x certification speedup over the certification that applies randomized smoothing of approximate model from scratch. We extensively evaluate the performance of IRS when applying several common DNN approximations such as pruning and quantization on state-of-the-art DNNs on CIFAR10 (Res Net-20, Res Net-110) and Image Net (Res Net-50) datasets. |
| Researcher Affiliation | Collaboration | 1University of Illinois Urbana-Champaign, 2VMware Research |
| Pseudocode | Yes | Algorithm 1 RS certification (Cohen et al., 2019) Inputs: f: DNN, σ: standard deviation, x: input to the DNN, n0: number of samples to predict the top class, n: number of samples for computing p A, α: confidence parameter... Algorithm 2 IRS algorithm: Certification with cache Inputs: f p: DNN obtained from approximating f, σ: standard deviation, x: input to the DNN, np: number of Gaussian samples used for certification, Cf: stores the information to be reused from certification of f, α and αζ: confidence parameters, γ: threshold hyperparameter to switch between estimation methods... Algorithm 3 Estimate ζx Inputs: f p: DNN obtained from approximating f, σ: standard deviation, x: input to the DNN, np: number of Gaussian samples used for estimating ζx, Cf: stores the information to be reused from certification of f, αζ: confidence parameter Output: Estimated value of ζx |
| Open Source Code | Yes | IRS code is available at https://github.com/uiuc-arc/Incremental-DNN-Verification. |
| Open Datasets | Yes | We evaluate IRS on CIFAR-10 (Krizhevsky et al.) and Image Net (Deng et al., 2009). |
| Dataset Splits | No | The paper states using CIFAR-10 and Image Net datasets and refers to a 'test set' and 'validation set images' but does not provide explicit details on the training, validation, or test splits (e.g., percentages, sample counts, or specific predefined splits). |
| Hardware Specification | Yes | We ran experiments on a 48-core Intel Xeon Silver 4214R CPU with 2 NVidia RTX A5000 GPUs. |
| Software Dependencies | Yes | IRS is implemented in Python and uses Py Torch 2.0.1. (Paszke et al., 2019). |
| Experiment Setup | Yes | Hyperparameters. We use confidence parameters α = 0.001 for the certification of g, and αζ = 0.001 for the estimation of ζx. To establish a fair comparison, we set the baseline confidence with αb = α + αζ = 0.002. This choice ensures that both the baseline and IRS, provide certified radii with equal confidence. We use grid search to choose an effective value for γ. A detailed description of our hyperparameter search and its results are described in Section 5.4. ... We use n = 105 samples for certification of g. For certifying gp, we consider np values from {5%, . . . 50%} of n and σ = 1. |