Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs
Authors: Advik Basani, Xiao Zhang
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through comprehensive experiments, we show that GASP can produce natural adversarial prompts, significantly improving jailbreak success over baselines, reducing training times, and accelerating inference speed, thus making it an efficient and scalable solution for red-teaming LLMs. |
| Researcher Affiliation | Academia | Advik Raj Basani Birla Institute of Technology and Science, Goa EMAIL Xiao Zhang CISPA Helmholtz Center for Information Security EMAIL |
| Pseudocode | Yes | Algorithms 1 and 2 in Appendix C show the pseudocode, which details how GASP is trained. |
| Open Source Code | Yes | The Adv Suffixes dataset, the implementation of GASP, and all our experiments are available as open-source code at https://github.com/Trust MLRG/GASP. |
| Open Datasets | Yes | The Adv Suffixes dataset, the implementation of GASP, and all our experiments are available as open-source code at https://github.com/Trust MLRG/GASP. |
| Dataset Splits | Yes | We use a 0.75/0.25 split of Adv Suffixes for pretraining and finetuning Suffix LLM (i.e., β = 0.75 in Algorithm 1). |
| Hardware Specification | Yes | For Suffix LLM, we use the LLama-3.1-8B-Lexi-Uncensored-V2 model and run all experiments on three NVIDIA DGX A100s (40GB). |
| Software Dependencies | No | The paper mentions models like LLama-3.1-8B-Lexi-Uncensored-V2, but does not list specific software dependencies with version numbers like Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | We use a 0.75/0.25 split of Adv Suffixes for pretraining and finetuning Suffix LLM (i.e., β = 0.75 in Algorithm 1). For reproducibility, we summarize all our training configurations and hyperparameters in Table 4. |