Adversarial Training Reduces Information and Improves Transferability
Authors: Matteo Terzi, Alessandro Achille, Marco Maggipinto, Gian Antonio Susto2674-2682
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our results employing robust networks trained on CIFAR-10, CIFAR-100 and Image Net on several datasets. Moreover, we show that Adversarial Training reduces Fisher information of representations about the input and of the weights about the task, and we provide a theoretical argument which explains the invertibility of deterministic networks without violating the principle of minimality. Finally, we leverage our theoretical insights to remarkably improve the quality of reconstructed images through inversion. |
| Researcher Affiliation | Collaboration | Matteo Terzi1, Alessandro Achille2, Marco Maggipinto1, Gian Antonio Susto1 1Department of Information Engineering, University of Padova 2AWS |
| Pseudocode | No | The paper describes methods using mathematical formulations and descriptive text, but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code for the methodology or links to a code repository. |
| Open Datasets | Yes | We employ CIFAR-10 (Krizhevsky, Nair, and Hinton 2009), CIFAR-100 (Krizhevsky, Nair, and Hinton 2009) and Image Net (Deng et al. 2009) as source datasets. |
| Dataset Splits | Yes | We employ CIFAR-10 (Krizhevsky, Nair, and Hinton 2009), CIFAR-100 (Krizhevsky, Nair, and Hinton 2009) and Image Net (Deng et al. 2009) as source datasets. All the experiments are obtained with Res Net-50 and ε = 1 for CIFAR and ε = 3 for Image Net as described in (Ilyas et al. 2019) and in the Appendix. In Table 1 we show performance of fine-tuning for the networks pretrained on CIFAR10 and CIFAR-100 transferring to CIFAR-10, CIFAR-100 F-MNIST (Xiao, Rasul, and Vollgraf 2017), MNIST (Le Cun and Cortes 2010) and SVHN (Netzer et al. 2011). |
| Hardware Specification | No | The paper does not specify any particular hardware (e.g., GPU models, CPU types, or cloud instances) used for running the experiments. |
| Software Dependencies | No | The paper mentions using DNNs and SGD, but does not provide specific software names with version numbers (e.g., PyTorch 1.x, TensorFlow 2.x, or specific library versions). |
| Experiment Setup | Yes | All the experiments are obtained with Res Net-50 and ε = 1 for CIFAR and ε = 3 for Image Net as described in (Ilyas et al. 2019) and in the Appendix. We trained robust models with different ε (with the same seed) to verify this claim: as reported in Figure 5, we discovered that it is true only for ε > 1, pointing out that there may exist two different regimes. At the beginning of optimization, we perturb weights with n N(0, Σ ) and solve the inversion with fw+ n. |