When does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?
Authors: Lijie Fan, Sijia Liu, Pin-Yu Chen, Gaoyuan Zhang, Chuang Gan
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | With a thorough experimental study, we demonstrate that ADVCL outperforms the state-of-the-art self-supervised robust learning methods across multiple datasets (CIFAR-10, CIFAR-100 and STL-10) and finetuning schemes (linear evaluation and full model finetuning). |
| Researcher Affiliation | Collaboration | Lijie Fan1, Sijia Liu2, Pin-Yu Chen3, Gaoyuan Zhang3, Chuang Gan3 1 Massachusetts Institute of Technology, 2 Michigan State University, 3 MIT-IBM Watson AI Lab, IBM Research |
| Pseudocode | No | The paper provides a pipeline diagram (Figure 2) but no structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at https://github.com/Lijie Fan/Adv CL. |
| Open Datasets | Yes | With a thorough experimental study, we demonstrate that ADVCL outperforms the state-of-the-art self-supervised robust learning methods across multiple datasets (CIFAR-10, CIFAR-100 and STL-10) and finetuning schemes (linear evaluation and full model finetuning). |
| Dataset Splits | No | The paper mentions training and testing, and discusses finetuning schemes, but does not explicitly describe a validation dataset split or how it was used. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU models, CPU types, memory amounts) used for running its experiments. It only mentions using ResNet-18 as an encoder architecture. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers (e.g., Python 3.8, PyTorch 1.9, CUDA 11.1) needed to replicate the experiment. |
| Experiment Setup | Yes | Unless specified otherwise, we use 5-step 1 projected gradient descent (PGD) with = 8/255 to generate perturbations during pretraining, and use Auto-Attack and 20-step 1 PGD with = 8/255 to generate perturbations in computing AA and RA at test time. |