GuardT2I: Defending Text-to-Image Models from Adversarial Prompts
Authors: Yijun Yang, Ruiyuan Gao, Xiao Yang, Jianyuan Zhong, Qiang Xu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our extensive experiments reveal that GUARDT2I outperforms leading commercial solutions like Open AI-Moderation and Microsoft Azure Moderator by a significant margin across diverse adversarial scenarios. |
| Researcher Affiliation | Academia | Yijun Yang1,2 , Ruiyuan Gao1, Xiao Yang2 , Jianyuan Zhong1, Qiang Xu1 1The Chinese University of Hong Kong, 2Tsinghua University {yjyang,rygao,jyzhong,qxu}@cse.cuhk.edu.hk,{yangyj16,yangxiao19}@tsinghua.org.cn |
| Pseudocode | Yes | Algorithm 1 Inference Workflow of GUARDT2I |
| Open Source Code | Yes | Our framework is available at https://github.com/cure-lab/Guard T2I. |
| Open Datasets | Yes | Training Dataset. LAION-COCO [40] represents a substantial dataset comprising 600M highquality captions that are paired with publicly sourced web images. This dataset encompasses a diverse range of prompts, including both standard and NSFW content, mirroring real-world scenarios. We use a subset of LAION-COCO consisting of 10M randomly sampled prompts to fine-tune our c LLM. |
| Dataset Splits | No | The paper mentions training and testing datasets, but does not explicitly specify a validation dataset split or its use. |
| Hardware Specification | Yes | We conduct our training and main experiments on the NVIDIA RTX4090 GPU with 24GB of memory. For adaptive attack and computational cost evaluation, we conduct experiments on the NVIDIA A800 GPU with 80 GB of memory. |
| Software Dependencies | No | The paper mentions using 'Sentence-transformer' and 'Adam optimizer' but does not specify version numbers for these or other key software libraries. |
| Experiment Setup | Yes | We implement c LLM with 24 transformer blocks. Its initial weights are sourced from [34]. ...We fine-tune c LLM using the Adam optimizer [31] with a learning rate of 1 × 10−5, and a batch size of 1024 for 50 epochs, using around 768 GPU hours on NVIDIA4090. |