A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models
Authors: Yihan Wu, Zhengmian Hu, Junfeng Guo, Hongyang Zhang, Heng Huang
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive empirical evaluation on various language models and tasks demonstrates our approach s distribution-preserving property, accessibility, and resilience, making it a effective solution for watermarking tasks that demand impeccable quality preservation. Our experimental section consists of five parts. |
| Researcher Affiliation | Academia | 1Department of Computer Science, University of Maryland College Park 2School of Computer Science, University of Waterloo. |
| Pseudocode | Yes | Algorithm 1 Di Pmark generator |
| Open Source Code | Yes | Code is available at1. 1https://github.com/yihwu/Di Pmark.git |
| Open Datasets | Yes | For the MT task, we focus on English-to Romanian translation. We employ the Multilingual BART (MBart) model (Liu et al., 2020) on the WMT 14 En-Ro corpus. For the TS task, we employ the BART-large model (Liu et al., 2020). For the TS task, our experimentation employs the BART-large model (Liu et al., 2020) in conjunction with the CNN-DM corpus (Hermann et al., 2015) as our designated testing dataset. |
| Dataset Splits | No | The paper mentions "1,999 examples in the test set" for WMT 14 and "11,490 examples" for CNN-DM (implied test set here) but doesn't explicitly state the train/validation splits or how these datasets were partitioned for those purposes. No explicit validation split information or full dataset splits are provided. |
| Hardware Specification | Yes | All experiments are conducted on three Nvidia A6000 GPUs with 48GB of memory. |
| Software Dependencies | No | The experiments are implemented using the Huggingface library (Wolf et al., 2019), a widely adopted platform for model development and sharing within the NLP community. No specific version numbers are provided for Huggingface or other key software dependencies. |
| Experiment Setup | Yes | Specifically for Di Pmark, we select values for α from the set {0.3, 0.35, 0.4, 0.45, 0.5}, while for the Soft watermark (Kirchenbauer et al., 2023), we choose green list bias values δ from the set {0.0, 1.0, 1.5, 2.0} alongside a fixed green list separator γ = 0.5, indicating that 50% of tokens are green while the remainder are red. Texture key generation relies on the most recent five tokens as texture key. |