Gated POS-Level Language Model for Authorship Verification
Authors: Linshu Ouyang, Yongzheng Zhang, Hui Liu, Yige Chen, Yipeng Wang
IJCAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experimental results show that our method achieves significantly better accuracy than state-of-the-art competing methods, especially in cross-topic scenarios (over 5% in terms of AUC-ROC). |
| Researcher Affiliation | Academia | Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | As shown in the Table 1, we used 4 publicly available authorship verification datasets, which were widely used by previous studies [Halvani et al., 2017; Bevendorff et al., 2019a; Bagnall, 2015; Halvani et al., 2018], with different genre and sizes. ...Based on these insights, they constructed an improved authorship verification corpus named Gutenberg [Bevendorff et al., 2019a]. ...The Enron data set comes from the public Enron emails dataset. ...The Reddit dataset is obtained from [Halvani et al., 2017]. |
| Dataset Splits | Yes | These data sets are respectively divided into two parts, the development set, and the test set. The PAN-15 and Gutenberg datasets have been pre-divided by the provider. The other two datasets are randomly divided according to the ratio of 1: 4 following the common practice in the research of authorship verification [Halvani et al., 2017]. |
| Hardware Specification | Yes | We implement the proposed method with Py Torch and run all the experiments on a GPU with 11GB memory. |
| Software Dependencies | Yes | We implement the proposed method with Py Torch...How to perform POS tagging is not the focus of this article, therefore we rather arbitrarily choose the popular open-source POS tagger Spacy [Matthew and Ines, 2015]. ...spacy: Industrial-strength nlp. https://spacy.io/, 2015. [Version=2.2.3]. |
| Experiment Setup | Yes | The most important hyperparameters of our model are the size of the embedding layer and the size of the hidden layer. These hyperparameters can be effectively selected on the development set and do not vary much on different datasets. Except for the Enron dataset where the best hidden size is 16, on other datasets the model achieves the best results on the development set when the hidden size is 64. In training, we follow the common practice to calculate the gradient with truncated backpropagation through time [Werbos and others, 1990; Graves, 2013]. Specifically, we truncated the gradient for 20 time-steps and perform gradient descent utilizing vanilla SGD optimizer combined with gradient clipping at 0.25. All models are trained for 100 epochs and the learning rate is set to 5. |