Protecting Intellectual Property of Language Generation APIs with Lexical Watermark
Authors: Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, Chenguang Wang10758-10766
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct intensive experiments on generation tasks, i.e., machine translation, text summarization, and image caption, to validate our approach. Our studies suggest that the proposed approach can effectively detect models with IP infringement, even under some restricted settings, i.e., cross-domain querying and mixture of watermarked and non-watermarked data |
| Researcher Affiliation | Collaboration | Xuanli He, 1 Monash University 2 The Australian National University 3 Sony AI 4 Microsoft Research Asia 5 UC Berkeley |
| Pseudocode | No | The paper describes algorithmic steps verbally and with mathematical formulas but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code and data are available at: https://github.com/xlhex/NLGapi_watermark.git |
| Open Datasets | Yes | We consider WMT14 German (De) English (En) translation (Bojar et al. 2014) as the testbed. ... We use CNN/DM dataset for the summarization task. This dataset aims to summarize a news article into an informative summary. We recycle the version preprocessed by See et al. (2017). ... We evaluate the proposed approach on MSCOCO data (Lin et al. 2014) and use the split provided by Karpathy et al. (2015). |
| Dataset Splits | Yes | Table 2: Statistics of datasets used in our experiments. Train Dev Test WMT14 4.5M 3K 200 CNN/DM 287K 13K 200 MSCOCO 567K 25K 200 |
| Hardware Specification | No | The computational resources of this work are supported by the Multi-modal Australian Science S Imaging and Visualisation Environment (MASSIVE) (www.massive.org.au). |
| Software Dependencies | No | Moses (Koehn et al. 2007) is used to pre-process all corpora, with all the text cased. ... we use Transformer as the backbone model. ... we consider using BART (Lewis et al. 2020) and m BART (Liu et al. 2020) for summarization and translation respectively. ... we first tag all English sentences from the training data with spa Cy POS tagger. |
| Experiment Setup | Yes | Both the victim model and the extracted model are trained with Transformer-base (Vaswani et al. 2017). Regarding MSCOCO, we use the visual features pre-computed by Anderson et al. (2018) as the inputs to the Transformer encoder. Since the 6-layer model is not converged for CNN/DM in the preliminary experiments, we reduced the number of layers to 3. A 32K BPE vocabulary (Sennrich, Haddow, and Birch 2016) is applied to WMT14 and CNN/DM, while 10K subword units is used for MSCOCO. |