Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
ADAPT to Robustify Prompt Tuning Vision Transformers
Authors: Masih Eskandar, Tooba Imtiaz, Zifeng Wang, Jennifer Dy
TMLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our methods on benchmark datasets with a focus on increasing adversarial robustness while keeping the number of tuned parameters relatively low. When using prompt tuning, we demonstrate the gradient obfuscation occurring in previous methods, and show that our proposed method overcomes this issue. Finally, we perform ablation studies on the different components of our method. |
| Researcher Affiliation | Academia | Masih Eskandar EMAIL Department of Electrical & Computer Engineering Northeastern University |
| Pseudocode | Yes | Algorithm 1 shows a pseudocode of our training algorithm. Algorithm 1 ADAPT training step Require: data pair x, y, frozen classifier f, tunable prompts θp, attack step size α, maximum perturbation magnitude ϵ, number of attack steps s, learning rate η x p x + U( ϵ, ϵ) U is the uniform noise distribution for i = 1 s do x p x p + α x p LCE(f(θp, x p), y) eq. (13) x p Project(x p, x ϵ, x + ϵ) Project x p onto the ϵ ball end for L LADAP T eq. (14) θp θp + η θp L Optimizer gradient step |
| Open Source Code | Yes | Our implementation is available on Github1. 1https://github.com/Gnomy17/robust_prompt |
| Open Datasets | Yes | Datasets. We perform experiments on CIFAR-10(Krizhevsky et al., a), CIFAR100 (Krizhevsky et al., b), and Imagenette (Howard). |
| Dataset Splits | Yes | Table 1: Test set accuracy of previous adversarial robustness methods combined with prompt tuning against unperturbed inputs, traditional adversarial attacks, PGD10 (multi-step) and FGSM (single step) as well as our adaptive attack. Table 2: Test set accuracy on the CIFAR10 dataset for different methods. |
| Hardware Specification | Yes | Each method was trained using an NVIDIA Tesla V100 SXM2. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | Each method in the prompt tuning scenario was trained for 20 epochs with a cyclic learning rate while each method in the fine tuning scenario was trained for 40 epochs with an annealing learning rate. |