Are Generative Classifiers More Robust to Adversarial Attacks?
Authors: Yingzhen Li, John Bradshaw, Yash Sharma
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results suggest that deep Bayes classifiers are more robust than deep discriminative classifiers, and that the proposed detection methods are effective against many recently proposed attacks. We evaluate the robustness of the proposed generative classifier on MNIST and a binary classification dataset derived from CIFAR-10. We further show the advantage of generative classifiers over a number of discriminative classifiers, including Bayesian neural networks and discriminative LVMs. |
| Researcher Affiliation | Collaboration | 1Microsoft Research Cambridge, UK 2University of Cambridge, UK 3Max Planck Institute for Intelligent Systems, Germany 4Eberhard Karls University of T ubingen, Germany. |
| Pseudocode | No | The paper does not contain any figures, blocks, or sections explicitly labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | Yes | We carry out a number of tests on the deep Bayes classifiers, our implementation is available at https://github. com/deepgenerativeclassifier/Deep Bayes. |
| Open Datasets | Yes | We evaluate the robustness of the proposed generative classifier on MNIST and a binary classification dataset derived from CIFAR-10. For MNIST tests, we use dim(z) = 64 for the LVM-based classifiers. We further consider the same set of evaluations on CIFAR-binary, a binary classification dataset containing airplane and frog images from CIFAR-10. |
| Dataset Splits | No | The paper mentions 'training data' and 'test dataset' with specific sizes (e.g., '10,000 datapoints' for MNIST test, '1,577 instances' for CIFAR-binary test), but it does not provide explicit details about a separate validation split or the percentages/counts for training, validation, and test sets. |
| Hardware Specification | No | The paper does not provide any specific details regarding the hardware (e.g., GPU models, CPU types, cloud instance specifications) used for running the experiments. |
| Software Dependencies | No | The paper mentions the 'Clever Hans 2.0 library' and references a 'pretrained VGG16 network', but it does not specify version numbers for general software dependencies such as programming languages, deep learning frameworks (e.g., PyTorch, TensorFlow), or specific libraries used for the implementation. |
| Experiment Setup | Yes | We use K = 10 Monte Carlo samples for all the classifiers. The constructed BNN has 2x more channels than LVM-based classifiers, making the comparison slightly unfair , as the BNN layers have more capacity. For MNIST tests, we use dim(z) = 64 for the LVM-based classifiers. The distortion strengths as ϵ {0.1, 0.2, 0.3, 0.4, 0.5} for MNIST, and ϵ {0.01, 0.02, 0.05, 0.1, 0.2} for CIFAR-binary. We use the SPSA ℓ attack (Uesato et al., 2018), which is similar to the white-box attacks, except that gradients are numerically estimated using the logit values from the victim classifier. We use fully-connected neural networks for these classifiers, and select from VGG16 the 9th convolution layer (CONV9) and the first fully connected layer after convolution (FC1) as the feature layers to ensure 90% test accuracy. |