GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks
Authors: Salah Ghamizi, Jingfeng Zhang, Maxime Cordy, Mike Papadakis, Masashi Sugiyama, Yves Le Traon
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments demonstrate that GAT outperforms eight state-of-the-art AT techniques based on data augmentation and training optimization, with an improvement on CIFAR10 of 3.14% to 26.4% compared to state of the art adversarial training with data-augmentation. GAT shines in scarce data scenarios (e.g. medical diagnosis tasks), where data augmentation is not applicable. Our large study across five datasets and six tasks demonstrates that task augmentation is an efficient alternative to data augmentation, and can be key to achieving both clean and robust performances. |
| Researcher Affiliation | Academia | 1The University of Luxembourg 2RIKEN Center for Advanced Intelligence Project (AIP) 3The University of Tokyo. Correspondence to: Jingfeng Zhang <jingfeng.zhang@riken.jp>. |
| Pseudocode | Yes | Algorithm 1 Pseudo-Algorithm of GAT; Algorithm 2 Pseudo-Algorithm of GAT; Algorithm 3 MGDA(θs, θt,l) procedure |
| Open Source Code | Yes | Our algorithm and replication packages are available on https://github.com/yamizi/taskaugment |
| Open Datasets | Yes | CIFAR-10 (Krizhevsky et al., 2009) is a 32x32 color image dataset. ... Che Xpert (Irvin et al., 2019) is a public chest X-ray dataset. |
| Dataset Splits | No | The paper mentions 'early stopping' which implies the use of a validation set, but it does not specify the exact percentages or sample counts for the training/validation/test splits. It only details subsets used for training scenarios (10%, 25%, 50%). |
| Hardware Specification | Yes | We train all our models on slurm nodes, using single node training. Each node has one A100 GPU 32Gb V100 SXM2. |
| Software Dependencies | No | Our license is MIT Licence, and we use the following external packages: Torchxrayvision: ... Apache Licence, Taskonomy/Taskgrouping: ... MIT Licence, Lib MTL: ... MIT Licence. The paper lists external packages but does not provide specific version numbers for them. |
| Experiment Setup | Yes | Both natural and AT is combined with common data augmentations (rotation, cropping, scaling), using SGD with lr=0.1, a cosine annealing, and early stopping. We train CIFAR-10 models for 400 epochs and Che Xpert models for 200 epochs. We perform AT following Madry s approach (Madry et al., 2017a) with a 10-steps PGD attack and ϵ = 8/255 size budgets, and we only target the main task to craft the adversarial examples. |