Uncovering Adversarial Risks of Test-Time Adaptation
Authors: Tong Wu, Feiran Jia, Xiangyu Qi, Jiachen T. Wang, Vikash Sehwag, Saeed Mahloujifar, Prateek Mittal
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through comprehensive evaluations, we demonstrate the high effectiveness of our attack on multiple benchmarks across six TTA methods. In response, we investigate two countermeasures to robustify the existing insecure TTA implementations, following the principle of security by design. Our Contributions. To exploit this vulnerability, we present a novel attack called Distribution Invading Attack (DIA), which exploits TTA by introducing malicious data (Section 4). We empirically illustrate that DIA achieves a high attack success rate (ASR) on various benchmarks, including CIFAR-10-C, CIFAR-100-C, and Image Net-C (Hendrycks & Dietterich, 2019) against a range of TTA methods, such as Te BN (Nado et al., 2020), TENT (Wang et al., 2021b), and Hard PL (Lee et al., 2013) in Section 5. |
| Researcher Affiliation | Academia | 1Princeton University 2Penn State University. |
| Pseudocode | Yes | Algorithm 1 for constructing Distribution Invading Attack |
| Open Source Code | Yes | The code is available at https://github. com/inspire-group/tta_risk |
| Open Datasets | Yes | We evaluate our attacks on well-established distribution shift benchmarks, namely CIFAR-10 to CIFAR-10-C, CIFAR-100 to CIFAR-100-C, and Image Net to Image Net-C (Hendrycks & Dietterich, 2019) |
| Dataset Splits | No | The paper defines evaluation sets and test data, but does not explicitly describe a separate 'validation' split or its size/percentage for the main experiments. It refers to 'test data' being processed batch by batch for TTA, which could serve a validation-like role in adaptation, but not in terms of traditional train/val/test dataset splitting for model development. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU specifications, or memory amounts used for running the experiments. It only mentions the models used (e.g., Res Net-26, Res Net-50, VGG-19, WRN-28). |
| Software Dependencies | No | The paper mentions optimizers and training parameters like 'SGD optimizer with a 0.1 learning rate, 0.9 momentum, and 0.0005 weight decay' and 'Adam for the TTA optimizer, η = 0.001 for the TTA learning rate, and 1 for temperature'. However, it does not specify software dependencies with version numbers such as Python, PyTorch, TensorFlow, CUDA, or specific library versions. |
| Experiment Setup | Yes | For training models on the CIFAR dataset, we use the SGD optimizer with a 0.1 learning rate, 0.9 momentum, and 0.0005 weight decay. We train the model with a batch size of 256 for 200 epochs. We also adjust the learning rate using a cosine annealing schedule (Loshchilov & Hutter, 2016). This shares the same configurations with Goyal et al. (2022). For the Image Net benchmark, we directly utilize the models downloaded from Robust Bench (Croce et al., 2020) (https://robustbench.github.io/). We use the same default hyperparameters with Wang et al. (2021b) and Goyal et al. (2022), where the code is available at github.com/Dequan Wang/tent and github.com/locuslab/tta-conjugate. Besides setting the batch size to 200, we use Adam for the TTA optimizer, η = 0.001 for the TTA learning rate, and 1 for temperature. TTA is done in 1 step for each test batch. For other hyperparameters, we set the attacking steps N = 500 and attacking optimization rate α = 1/255. |