Certified Defense to Image Transformations via Randomized Smoothing
Authors: Marc Fischer, Maximilian Baader, Martin Vechev
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | A thorough evaluation of all methods on common image datasets, achieving provable distributional robust accuracy of 73% for rotations with up to 30 on Restricted Image Net. |
| Researcher Affiliation | Academia | Marc Fischer, Maximilian Baader, Martin Vechev Department of Computer Science ETH Zurich {marc.fischer, mbaader, martin.vechev}@inf.ethz.ch |
| Pseudocode | No | The paper describes methods and steps in prose but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | We provide an implementation of all methods at https://github.com/eth-sri/transformation-smoothing. |
| Open Datasets | Yes | We evaluate on Image Net [30], Restricted Image Net (RImage Net)[31], a subset of Image Net with 10 classes, CIFAR-10 [32], and MNIST [33]. |
| Dataset Splits | No | The paper mentions using a 'training dataset' and evaluating on 'test set' but does not provide explicit train/validation/test split percentages, absolute sample counts, or detailed methodology for dataset partitioning to ensure reproducibility of splits. |
| Hardware Specification | Yes | All experiments were performed on a machine with 2 Ge Force RTX 2080 Tis and an Intel(R) Core(TM) i9-9900K CPU. |
| Software Dependencies | No | The paper mentions using 'Py Torch [28]' and 'robustness [29]' but does not provide specific version numbers for these software components or other key dependencies. |
| Experiment Setup | Yes | Further, we let σγ, αγ, nγ, rγ and σδ, αδ, nδ, rδ denote the parameters and radius required to use Theorem 3.2 and Theorem 3.1 in practice, respectively. [...] Here we use αδ = 0.002 and αγ = 0.01 for confidences. [...] σδ = 0.3 for MNIST, σδ = 0.25 for CIFAR-10 and σδ = 0.5 for (R)Image Net. [...] For rotations (Γ = 10, σγ = 30, nγ = 2000, 3 attacks per image, 1000 images) we fix E = 0.7 and use 100 samples of β to obtain the correct αE (Eq. (9)). [...] We use 10 refinement steps and nδ = 200 for both rotations and translations. |