Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning
Authors: Rundong Luo, Yifei Wang, Yisen Wang
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experiments, we show that DYNACL can improve state-of-the-art self-AT robustness by 8.84% under Auto-Attack on the CIFAR-10 dataset, and can even outperform vanilla supervised adversarial training for the first time. |
| Researcher Affiliation | Academia | Rundong Luo1 Yifei Wang2 Yisen Wang3,4 1School of EECS, Peking University 2School of Mathematical Sciences, Peking University 3National Key Lab. of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University 4Institute for Artificial Intelligence, Peking University |
| Pseudocode | Yes | Algorithm 1 demonstrates the pseudo-code of our proposed DYNACL and DYNACL++. |
| Open Source Code | Yes | Our code is available at https://github.com/PKU-ML/DYNACL. |
| Open Datasets | Yes | In this section, we evaluate DYNACL and DYNACL++ under the benchmark datasets: CIFAR-10, CIFAR-100 (Krizhevsky, 2009), and STL-10 (Coates et al., 2011) |
| Dataset Splits | No | The paper explicitly mentions 'training set' and 'test set' but does not specify a separate 'validation set' or how it was used for hyperparameter tuning. While it uses standard datasets with predefined train/test splits, a distinct validation split is not detailed. |
| Hardware Specification | Yes | We evaluate the total training time of self-AT methods with a single RTX 3090 GPU. |
| Software Dependencies | No | The paper mentions 'torchvision', 'transforms', and 'Py Torch-style code' but does not specify version numbers for these software components, which is required for reproducibility. |
| Experiment Setup | Yes | We set the decay period K = 50, and reweighting rate λ = 2/3. As for other parts, we mainly follow the training recipe of ACL (Jiang et al., 2020)... Specifically, we train the backbone for 1000 epochs with LARS optimizer and cosine learning rate scheduler... we tune the classification head only by standard training for 10 epochs, then we tune both the encoder and the classification head by TRADES (Zhang et al., 2019) for 25 epochs... As for SLF and ALF we train the linear classifier for 25 epochs with an initial learning rate of 0.01 on CIFAR-10 and 0.1 on CIFAR-100 and STL-10 (decays at the 10th and 20th epoch by 0.1) and batch size 512. For AFF, we employ the TRADES (Zhang et al., 2019) loss with default parameters... Finetuning lasts 25 epochs with an initial learning rate of 0.1 (decays by 0.1 at the 15th and 20th epoch) and batch size 128. |