Generalizable Adversarial Training via Spectral Normalization
Authors: Farzan Farnia, Jesse Zhang, David Tse
ICLR 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we extend the notion of margin loss to adversarial settings and bound the generalization error for DNNs trained under several well-known gradient-based attack schemes, motivating an effective regularization scheme based on spectral normalization of the DNN s weight matrices. We also provide a computationally-efficient method for normalizing the spectral norm of convolutional layers with arbitrary stride and padding schemes in deep convolutional networks. We evaluate the power of spectral normalization extensively on combinations of datasets, network architectures, and adversarial training schemes. |
| Researcher Affiliation | Academia | Farzan Farnia , Jesse M. Zhang , David N. Tse Department of Electrical Engineering Stanford University {farnia,jessez,dntse}@stanford.edu |
| Pseudocode | Yes | Algorithm 1 Convolutional power iteration |
| Open Source Code | No | The code will be made readily available. |
| Open Datasets | Yes | The datasets we evaluate are CIFAR10, MNIST, and SVHN. |
| Dataset Splits | Yes | For each experiment, we cross validate 4 to 6 values of β (see (9)) using a fixed validation set of 500 samples. |
| Hardware Specification | Yes | These ratios were obtained by running the experiments on one NVIDIA Titan Xp GPU for 40 epochs. |
| Software Dependencies | No | All experiments are implemented in Tensor Flow (Abadi et al., 2016). No specific version number is provided for Tensor Flow or other software libraries. |
| Experiment Setup | Yes | For PGM, we used r = 15 iterations and α = 2ϵ/r. Additionally, for FGM and PGM we used ℓ2-type attacks (unless specified) with magnitude ϵ = 0.05E ˆ P [ X 2] (this value was approximately 2.44 for CIFAR10). For WRM, we implemented gradient ascent as discussed by Sinha et al. (2018). Additionally, for WRM training we used a Lagrangian coefficient of 0.002E ˆ P [ X 2] for CIFAR10 and SVHN and a Lagrangian coefficient of 0.04E ˆ P [ X 2] for MNIST in a similar manner to Sinha et al. (2018). |