Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMs
Authors: Ran Li, Hao Wang, Chengzhi Mao
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On standard benchmarks like Adv Bench and Jailbreak Bench, LARGO surpasses leading jailbreaking techniques, including Auto DAN, by 44 points in attack success rate. Our findings demonstrate a potent alternative to agentic LLM prompting, highlighting the efficacy of interpreting and attacking LLM internals through gradient optimization. |
| Researcher Affiliation | Academia | Ran Li Columbia University EMAIL Hao Wang Rutgers University EMAIL Chengzhi Mao Rutgers University EMAIL |
| Pseudocode | Yes | Algorithm 1 Adversarial Suffix Generation Require: query q, target sequence y , suffix length L, embedding matrix Emb, max iterations T |
| Open Source Code | Yes | Our code is available at https://github.com/ranhli/LARGO. |
| Open Datasets | Yes | We evaluate LARGO on Adv Bench (Zou et al., 2023) and Jailbreak Bench (Chao et al., 2024). Both are comprehensive benchmarks aimed at evaluating the vulnerability of LLMs to adversarial inputs, including harmful or toxic behavior... Both are distributed under a MIT license. |
| Dataset Splits | Yes | In our experiments, we utilize a random sample of 200 harmful behavior instructions from each dataset. |
| Hardware Specification | Yes | We conduct all experiments on a NVIDIA H100 GPU with 80GB VRAM. |
| Software Dependencies | No | We utilize mixed-precision training (bfloat16) and optimize the suffix latent using the Adam optimizer with a learning rate of 1 × 10−3 and weight decay of 0.001. This text does not specify software library names with version numbers. |
| Experiment Setup | Yes | We utilize mixed-precision training (bfloat16) and optimize the suffix latent using the Adam optimizer with a learning rate of 1 × 10−3 and weight decay of 0.001. For all experiments, we set the suffix length of our attack to 200 and the max number of refinement iterations to 15. For each baseline attack, we use the publicly-available implementation with the identical suffix length and search iterations. For all jailbreaking tests, we set model temperature to 0 for deterministic and reproducible results. |