Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
Authors: Matthias Hein, Maksym Andriushchenko
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experiments The goal of the experiments is the evaluation of the robustness of the resulting classifiers and not necessarily state-of-the-art results in terms of test error. In all cases we compute the robustness guarantees from Theorem 2.1 (lower bound on the norm of the minimal change required to change the classifier decision), where we optimize over R using binary search, and adversarial samples with the algorithm for the 2-norm from Section 4 (upper bound on the norm of the minimal change required to change the classifier decision), where we do a binary search in the classifier output difference in order to find a point on the decision boundary. Additional experiments can be found in the supplementary material. |
| Researcher Affiliation | Academia | Matthias Hein and Maksym Andriushchenko Department of Mathematics and Computer Science Saarland University, Saarbrücken Informatics Campus, Germany |
| Pseudocode | No | The paper states 'proofs and algorithms are in the supplement' in Section 4, but no pseudocode or algorithm blocks are provided within the main body of the paper. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described in this paper. |
| Open Datasets | Yes | Kernel methods: ... We show the results for MNIST (60000 training and 10000 test samples). ... Neural Networks: We do experiments for MNIST and CIFAR10 in three settings: plain, data augmentation and adversarial training. |
| Dataset Splits | Yes | Kernel methods: ... We show the results for MNIST (60000 training and 10000 test samples). However, we have checked that parameter selection using a subset of 50000 images from the training set and evaluating on the rest yields indeed the parameters which give the best test errors when trained on the full set. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | Yes | Neural Networks: We use a one hidden layer network with 1024 hidden units and the softplus activation function with α = 10. Thus the resulting classifier is continuously differentiable. We compare three different regularization techniques: weight decay, dropout and our Cross Lipschitz regularization. Training is done with SGD. For each method we have adapted the learning rate (two per method) and regularization parameters (4 per method) so that all methods achieve good performance. |