DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection
Authors: XIAO YU, Yuang Qi, Kejiang Chen, Guoqiang Chen, Xi Yang, Pengyuan Zhu, Xiuwei Shang, Weiming Zhang, Nenghai Yu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Compared to the baselines, DPIC has achieved an average improvement of 6.76% and 2.91% in detecting texts from different domains generated by GPT-4 and Claude3, respectively. |
| Researcher Affiliation | Collaboration | 1University of Science and Technology of China, China 2Key Laboratory of Cyberspace Security, Ministry of Education, China 3Hefei High-dimensional Data Technology, China |
| Pseudocode | No | No explicit pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | Yes | We provide open access to the data and code, we also provided our model. |
| Open Datasets | Yes | In this paper, we used the open-source Human-Chat GPT Comparison Corpus (HC3) [14] dataset collected by previous researchers as a training set to ensure the reproducibility of our approach. |
| Dataset Splits | No | The paper mentions using the HC3 dataset as a “training set” and other datasets for “testing” but does not specify any validation set or its split percentage. |
| Hardware Specification | No | The paper states: “Our method does require more memory and time, compared to those that only input candidate text for classification, primarily due to the regeneration component involving the LLM. Detailed information regarding this can be found in the Appendix A.” However, the provided text does not contain specific hardware details such as GPU/CPU models or memory amounts. Appendix A was not provided. |
| Software Dependencies | Yes | Specifically, we use gte-Qwen1.5-7B-instruct as the encoder which can encode texts with a maximum of 32K tokens into embeddings of 4096 dimensions... We select two auxiliary models for the decoupling process, including Chat GPT and Vicuna-7b-v1.5. |
| Experiment Setup | Yes | Specifically, we use gte-Qwen1.5-7B-instruct as the encoder which can encode texts with a maximum of 32K tokens into embeddings of 4096 dimensions, while the classifier consists of three fully connected layers with ReLU function. The dimensions of the intermediate layers in the classifier are 1024 and 512, respectively. We freeze the weights of the Siamese encoder and train the classifier using binary cross-entropy loss. |