Theoretical evidence for adversarial robustness through randomization
Authors: Rafael Pinot, Laurent Meunier, Alexandre Araujo, Hisashi Kashima, Florian Yger, Cedric Gouy-Pailler, Jamal Atif
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper investigates the theory of robustness against adversarial attacks. It focuses on the family of randomization techniques that consist in injecting noise in the network at inference time. These techniques have proven effective in many contexts, but lack theoretical arguments. We close this gap by presenting a theoretical analysis of these approaches, hence explaining why they perform well in practice. More precisely, we make two new contributions. The first one relates the randomization rate to robustness to adversarial attacks. This result applies for the general family of exponential distributions, and thus extends and unifies the previous approaches. The second contribution consists in devising a new upper bound on the adversarial risk gap of randomized neural networks. We support our theoretical claims with a set of experiments. |
| Researcher Affiliation | Collaboration | Rafael Pinot1,2 Laurent Meunier1,3 Alexandre Araujo1,4 Hisashi Kashima5,6 Florian Yger1 Cédric Gouy-Pailler2 Jamal Atif1 1Université Paris-Dauphine, PSL Research University, CNRS, LAMSADE, Paris, France 2Institut LIST, CEA, Université Paris-Saclay 3Facebook AI Research, Paris, France 4Wavestone, Paris, France 5Kyoto University, Kyoto, Japan 6RIKEN Center for AIP, Japan |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide an unambiguous statement about releasing source code for the described methodology or a direct link to a code repository. |
| Open Datasets | Yes | We present our results and analysis on CIFAR-10, CIFAR-100 [22] and Image Net datasets [11]. |
| Dataset Splits | Yes | We train all networks for 200 epochs, a batch size of 400, dropout 0.3 and Leaky Relu activation with a slope on R of 0.1. We minimize the Cross Entropy Loss with Momentum 0.9 and use a piecewise constant learning rate of 0.1, 0.02, 0.004 and 0.00008 after respectively 7500, 15000 and 20000 steps. The networks achieve for CIFAR10 and 100 a TOP-1 accuracy of 95.8% and 79.1% respectively on test images. For the training of Image Net, we use the same hyper parameters setting as the original implementation. We train the network for 120 epochs with a batch size of 256, dropout 0.8 and Relu as activation function. All evaluations were done with a single crop on the non-blacklisted subset of the validation set. |
| Hardware Specification | No | This work was granted access to the Open POWER prototype from GENCI-IDRIS under the Preparatory Access AP010610510 made by GENCI. |
| Software Dependencies | No | The paper mentions software tools like Wide Res Net, Inception Res Net v2, Cross Entropy Loss, and specific attacks (EAD, C&W, PGD), but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | We train all networks for 200 epochs, a batch size of 400, dropout 0.3 and Leaky Relu activation with a slope on R of 0.1. We minimize the Cross Entropy Loss with Momentum 0.9 and use a piecewise constant learning rate of 0.1, 0.02, 0.004 and 0.00008 after respectively 7500, 15000 and 20000 steps. The networks achieve for CIFAR10 and 100 a TOP-1 accuracy of 95.8% and 79.1% respectively on test images. For the training of Image Net, we use the same hyper parameters setting as the original implementation. We train the network for 120 epochs with a batch size of 256, dropout 0.8 and Relu as activation function. |