Punctuation-level Attack: Single-shot and Single Punctuation Can Fool Text Models
Authors: wenqiang wang, Chongyang Du, Tao Wang, Kaihao Zhang, Wenhan Luo, Lin Ma, Wei Liu, Xiaochun Cao
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on public datasets and SOTA models demonstrate the effectiveness of the punctuation attack and the proposed TPPE. We additionally apply the single punctuation attack to summarization, semantic-similarity-scoring, and text-to-image tasks, and achieve encouraging results. |
| Researcher Affiliation | Collaboration | 1Shenzhen Campus of Sun Yat-sen University 2Nanjing University 3Australian National University 4Meituan 5Tencent |
| Pseudocode | Yes | The specific pseudo code of TPPE and TPPEP are presented in Alg. 1 and Alg. 2 in the Appendix. |
| Open Source Code | No | The paper does not contain any explicit statements about the release of open-source code for the methodology described, nor does it provide a direct link to a code repository. |
| Open Datasets | Yes | For our TC task, we have opted to utilize the Co LA dataset [43]... Our choice for the paraphrasing task is the QQP dataset [21]. Furthermore, the NLI task involves predicting the relationship between pairs of input sentences and we employ the Wanli dataset [25]... Datasets from STS-B [26] and gigaword [20] datasets were chosen for the sss and summarization tasks. |
| Dataset Splits | No | While Table 4 shows columns for 'train' and 'test' subsets, the paper does not explicitly provide details about the training/validation/test dataset splits such as percentages, sample counts, or the methodology for creating these splits. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments, such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | The paper mentions various deep learning models used (e.g., ELECTRA, XLMR, Distil BERT, Ro BERTa, De BERTa, Stable Diffusion V2) but does not provide specific version numbers for any ancillary software dependencies like programming languages or libraries. |
| Experiment Setup | No | The paper describes the general experimental setting, such as focusing on black-box attacks and single punctuation perturbations, but it does not provide specific hyperparameters (e.g., learning rate, batch size, number of epochs) or detailed system-level training settings needed for reproduction. |