Adaptive Text Watermark for Large Language Models
Authors: Yepeng Liu, Yuheng Bu
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments over various language models (OPT-6.7B (Zhang et al., 2022), GPT-J-6B (Wang & Komatsuzaki, 2021), and Mistral-7B (Jiang et al., 2023)) to show the effectiveness of our method. |
| Researcher Affiliation | Academia | 1University of Florida. Correspondence to: Yepeng Liu <yepeng.liu@ufl.edu>, Yuheng Bu <buyuheng@ufl.edu>. |
| Pseudocode | Yes | Algorithm 1 Semantic-based Logits Scaling Vector Extraction (SLSVE) and Algorithm 2 Adaptive Watermark Generation |
| Open Source Code | Yes | Our code is available at https://github.com/yepengliu/adaptive-text-watermark.git. |
| Open Datasets | Yes | To test the performance of our methods, the results on realnewslike subset in C4 dataset (Raffel et al., 2020) are provided here, and additional results on ELI5 (Fan et al., 2019) can be found in Appendix B.2. |
| Dataset Splits | No | The paper describes how prompts and generated text are derived from C4 and ELI5 datasets but does not provide explicit train/validation/test dataset splits with percentages or sample counts for the evaluation of their watermarking method. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., GPU models, CPU models, or cloud instance types) used to run its experiments. |
| Software Dependencies | No | The paper mentions using 'Sentence-Transformers' and 'all-mpnet-base-v2' but does not provide specific version numbers for these software components or other dependencies. |
| Experiment Setup | Yes | In our watermarking algorithm, the default hyperparameters are set as follows: α = 2, M = 50. At each time step, the next token is sampled from the distribution using a combination of Top-K and Top-p sampling methods, with K = 50 and p = 0.9. For example, we set δ = 1.5 for OPT-6.7B and set δ = 0.6 for Mistral-7B. |