Effective Targeted Attacks for Adversarial Self-Supervised Learning
Authors: Minseon Kim, Hyeonjeong Ha, Sooel Son, Sung Ju Hwang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our method demonstrates significant enhancements in robustness when applied to noncontrastive SSL frameworks, and less but consistent robustness improvements with contrastive SSL frameworks, on the benchmark datasets. ... In this section, we extensively evaluate the efficacy of TARO with both contrastive and positive-pair only adversarial SSL frameworks. First, we compare the performance of our model to previous adversarial SSL methods that do not utilize any targeted attacks in Section 4.1. |
| Researcher Affiliation | Collaboration | Minseon Kim1, Hyeonjeong Ha1, Sooel Son1, Sung Ju Hwang1,2 1Korea Advanced Institute of Science and Technology (KAIST), 2Deep Auto.ai |
| Pseudocode | Yes | Algorithm 1 Targeted Attack Robust Self-Supervised Learning (TARO) for contrastive-based SSL |
| Open Source Code | Yes | Code is available in https://github.com/Kim-Minseon/TARO.git |
| Open Datasets | Yes | All models use the Res Net18 backbones that are trained on CIFAR-10 and CIFAR-100 with ℓ PGD attacks with the attack step of 10 and epsilon 8/255. |
| Dataset Splits | Yes | We evaluate the quality of the learned representations with the SSL frameworks, we utilize linear and robust linear evaluation, as shown in Table 3. Then, we validate the generality of TARO to contrastive-based adversarial SSL frameworks (Table 5). |
| Hardware Specification | Yes | All experiments are conducted with a two NVIDIA RTX 2080 Ti, except for the experiments with CIFAR-100 experiments. For CIFAR-100 experiments, two NVIDIA RTX 3080 are used. All experiments are processed in Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz. |
| Software Dependencies | No | The paper mentions using specific models like 'Res Net18' and optimizers like 'SGD optimizer' and 'LARS optimizer', but it does not list specific software dependencies (e.g., Python, PyTorch, TensorFlow) along with their version numbers required for reproduction. |
| Experiment Setup | Yes | For all methods, we train on Res Net18 [18] with ℓ attacks with attack strength of ϵ = 8/255 and step size of α = 2/255, with the number of inner maximization iterations set to K = 10. For the optimization, we train every model for 800 epochs using the SGD optimizer with the learning rate of 0.05, weight decay of 5e 4, and the momentum of 0.9. For data augmentation, we use a random crop with 0.08 to 1.0 size, horizontal flip with a probability of 0.5, color jitter with a probability of 0.8, and grayscale with a probability of 0.2. We exclude normalization for adversarial training. We set the weight of adversarial similarity loss w as 2.0. We use batch size 512 with two GPUs. |