PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalization
Authors: Jiancong Xiao, Ruoyu Sun, Zhi-Quan Luo
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper focuses on norm-based complexity, based on a PAC-Bayes approach (Neyshabur et al., 2017b). The main challenge lies in extending the key ingredient, which is a weight perturbation bound in standard settings, to the robust settings. Existing attempts heavily rely on additional strong assumptions, leading to loose bounds. In this paper, we address this issue and provide a spectrally-normalized robust generalization bound for DNNs. Compared to existing bounds, our bound offers two significant advantages: Firstly, it does not depend on additional assumptions. Secondly, it is considerably tighter, aligning with the bounds of standard generalization. Therefore, our result provides a different perspective on understanding robust generalization: The mismatch terms between standard and robust generalization bounds shown in previous studies do not contribute to the poor robust generalization. Instead, these disparities solely due to mathematical issues. Finally, we extend the main result to adversarial robustness against general non-ℓp attacks and other neural network architectures. |
| Researcher Affiliation | Academia | 1University of Pennsylvania, PA, USA 2Scool of Data Science, The Chinese University of Hong Kong, Shenzhen, China 3Shenzhen International Center for Industrial and Applied Mathematcs 4Shenzhen Research Institute of Big Data |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not explicitly state that the authors' own source code for the described methodology is publicly available, nor does it provide a direct link to a code repository for their work. Footnotes refer to GitHub links for settings of other works. |
| Open Datasets | Yes | We conducted experiments training MNIST, CIFAR-10, and CIFAR-100 datasets on VGG networks, see Appendix C. |
| Dataset Splits | No | The paper mentions training on MNIST, CIFAR-10, and CIFAR-100 datasets but does not explicitly provide details about training, validation, or test splits. It refers to 'following the training parameters described in (Neyshabur et al., 2017a)' without specifying the split ratios or methods within this paper. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. It only refers to training on 'VGG networks' and following 'training parameters' from other works. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions, or specific libraries with their versions). |
| Experiment Setup | No | The paper states that 'We conducted experiments training MNIST, CIFAR-10, and CIFAR-100 datasets on VGG networks, following the training parameters described in (Neyshabur et al., 2017a)'. It refers to external sources for training settings rather than detailing specific hyperparameters or training configurations within the paper itself. |