Certifiably Adversarially Robust Detection of Out-of-Distribution Data
Authors: Julian Bitterwolf, Alexander Meinke, Matthias Hein
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide experimental results for image recognition tasks with MNIST [22], SVHN [30] and CIFAR-10 [21] as in-distribution datasets. |
| Researcher Affiliation | Academia | Julian Bitterwolf University of Tübingen Alexander Meinke University of Tübingen Matthias Hein University of Tübingen |
| Pseudocode | No | The paper contains mathematical equations and descriptions of the method but no explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available under https://gitlab.com/Bitterwolf/GOOD. |
| Open Datasets | Yes | We provide experimental results for image recognition tasks with MNIST [22], SVHN [30] and CIFAR-10 [21] as in-distribution datasets. ... For the training out-distribution, we use 80 Million Tiny Images (80M) [37]. ... As OOD evaluation sets we use Fashion MNIST [39], the Letters of EMNIST [5], grayscale CIFAR-10, and Uniform Noise for MNIST, and CIFAR-100 [21], CIFAR-10/SVHN, LSUN Classroom [40], and Uniform Noise for SVHN/CIFAR-10. |
| Dataset Splits | No | The paper mentions training and test sets but does not explicitly describe a separate validation dataset split or how it was used for hyperparameter tuning. Standard splits are implied for the well-known datasets, but no specific validation split details are provided. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models, or cloud computing instance specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions software components like ReLU, softmax, Adam, Auto-PGD, and scikit-learn, but does not specify version numbers for any of them. |
| Experiment Setup | Yes | For MNIST, we use the large architecture from [13], and for SVHN and CIFAR-10 a similar but deeper and wider model. The layer structure is laid out in Table 2 in the appendix. Data augmentation is applied to both inand out-distribution images during training. For MNIST we use random crops to size 28 28 with padding 4 and for SVHN and CIFAR-10 random crops with padding 4 as well as the quite aggressive augmentation Auto Augment [9]. Additionally, we apply random horizontal flips for CIFAR-10. ... we use linear ramp up schedules for ϵ and κ, which are detailed in Appendix D. As radii for the l -perturbation model on the out-distribution we use ϵ = 0.3 for MNIST, ϵ = 0.03 for SVHN and ϵ = 0.01 for CIFAR-10. As parameter κ for the trade-off between cross-entropy loss and the GOOD regularizer in (9) and (10), we set κ = 0.3 for MNIST and κ = 1 for SVHN and CIFAR-10. A training batch consists of 128 inand 128 out-distribution samples. |