Double Sampling Randomized Smoothing
Authors: Linyi Li, Jiawei Zhang, Tao Xie, Bo Li
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on MNIST, CIFAR-10, and Image Net verify our theory and show that DSRS certifies larger robust radii than existing baselines consistently under different settings. |
| Researcher Affiliation | Academia | 1University of Illinois Urbana-Champaign, Illinois, USA 2Peking University, Beijing, China. |
| Pseudocode | Yes | The detailed algorithm can be found in Alg. 2 in Appendix E.1: 1) we first perform a binary search on the certified radius r to determine the maximum radius that we can certify; 2) for current r, we determine the smoothed prediction confidence PA and QA from the confidence intervals of predicting the true class ( 5.2); 3) then, for current r we solve the dual problem by quick binary search for dual variables λ1 and λ2 (see Eqn. (10)) along with numerical integration ( 5.3). To guarantee the soundness of numerical-integration-based certification, we take the maximum possible error into account during the binary search. We will discuss further extensions in 5.4. |
| Open Source Code | Yes | Code is available at https: //github.com/llylly/DSRS. |
| Open Datasets | Yes | Extensive experiments on MNIST, CIFAR-10, and Image Net verify our theory and show that DSRS certifies larger robust radii than existing baselines consistently under different settings. |
| Dataset Splits | No | The paper mentions training models and evaluating on a 'test set', but it does not explicitly provide the specific percentages or counts for training, validation, and test splits for the datasets used. |
| Hardware Specification | No | The paper mentions running experiments and training models but does not provide specific hardware details such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | We implement DSRS in Python with about two thousand lines of code. The tool uses Py Torch2 to query a given base classifier with Monte-Carlo sampling in order to derive the confidence intervals [PA, PA] and [QA, QA]. Then, the tool builds the whole DSRS pipeline on Sci Py3 and Num Py4. The paper mentions software like Python, PyTorch, SciPy, and NumPy, but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | On MNIST and CIFAR-10, for all training methods, we train for 150 epochs. The learning rate is 0.01 and is decayed by 0.1 at the 50th and 100th epoch. For Consistency training, the hyperparameter λ = 5 on MNIST and λ = 20 on CIFAR-10. We use two noise vectors per training batch per instance. These are the best hyperparameters reported in (Jeong & Shin, 2020). The batch size is 256 on both MNIST and CIFAR-10 following (Jeong & Shin, 2020). For Smooth Mix training, we directly use the best hyperparameters from their open-source repository: https://github.com/jh-jeong/smoothmix/ blob/main/EXPERIMENTS.MD. |