Indiscriminate Poisoning Attacks on Unsupervised Contrastive Learning
Authors: Hao He, Kaiwen Zha, Dina Katabi
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically show that Contrastive Poisoning, not only drastically reduces the performance of CL algorithms, but also attacks supervised learning models, making it the most generalizable indiscriminate poisoning attack. We also show that CL algorithms with a momentum encoder are more robust to indiscriminate poisoning, and propose a new countermeasure based on matrix completion. Code is available at: https://github.com/kaiwenzha/contrastive-poisoning. |
| Researcher Affiliation | Academia | Hao He , Kaiwen Zha , Dina Katabi Computer Science and Artificial Intelligence Lab Massachusetts Institute of Technology {haohe,kzha,dk}@mit.edu |
| Pseudocode | Yes | Algorithm 1 Contrastive Poisoning |
| Open Source Code | Yes | Code is available at: https://github.com/kaiwenzha/contrastive-poisoning. |
| Open Datasets | Yes | We evaluate contrastive poisoning (CP) on multiple benchmark datasets: CIFAR-10/-100 (Krizhevsky et al., 2009), STL-10 (Coates et al., 2011), and Image Net-100. Image Net-100 is a randomly selected 100-class subset of the Image Net ILSVRC-2012 dataset (Russakovsky et al., 2015), containing 131.7K images. |
| Dataset Splits | No | For CIFAR-10/100, we use 50K for both pre-training the encoder and training the linear classifier, and 10K to test the linear classifier. For Image Net-100, we use 126.7K for training and 5K for testing. For STL-10, we only use it for linear probing. It has a standard split of 5K labeled trainset and 8K testset. The paper provides train and test set sizes but does not explicitly mention a separate validation split with specific details for all datasets. |
| Hardware Specification | Yes | Hardwares. We run CIFAR-10/-100 experiments on 4 NVIDIA TITAN Xp GPUs. We run Image Net100 experiments on 4 NVIDIA Tesla V100 GPUs. |
| Software Dependencies | No | The paper mentions software like Kornia, PyTorch, and PIL, but does not provide specific version numbers for any of them (e.g., 'PyTorch 1.9'). |
| Experiment Setup | Yes | Table 9: Hyper-parameters for different contrastive learning algorithms in our experiments. Sim CLR Mo Co v2 BYOL Optimizer SGD SGD SGD Weight Decay 10 4 10 4 10 4 Learning Rate (LR) 0.5 0.3 1.0 LR Scheduler Cosine Cosine Cosine Encoder Momentum 0.99 0.999 Loss function Info NCE Info NCE MSE Info NCE temperature 0.5 0.2 - For CIFAR-10/-100, the models are trained for 1000 epochs with a batch size of 512. For Image Net-100, the models are trained for 200 epochs with a batch size of 128. For sample-wise CP, T = 600, Tθ = 100, Tδ = 100, Tp = 5. For class-wise CP, T = 200, Tθ = 20, Tδ = 20, Tp = 1. In both attacks, we set the PGD learning rate ηδ to one tenth of the radius of the L ball, i.e., ηδ = ϵ/10 = 0.8/255. |