Fact-Enhanced Synthetic News Generation
Authors: Kai Shu, Yichuan Li, Kaize Ding, Huan Liu13825-13833
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiment results on real-world datasets demonstrate that the generated news contents of FACTGEN are consistent and contain rich facts. We also discuss an effective defending technique to identify these synthetic news pieces if FACTGEN was used to generate fake news. |
| Researcher Affiliation | Academia | Kai Shu 1, Yichuan Li 2, Kaize Ding3 and Huan Liu3 1 Illinois Institute of Technology, Chicago, IL, USA 2 Worcester Polytechnic Institute, Worcester, MA, USA 3 Arizona State University, Tempe, AZ, USA |
| Pseudocode | Yes | Algorithm 1 Training Procedure of FACTGEN |
| Open Source Code | Yes | The code is available at https://github.com/bigheiniu/FactGen |
| Open Datasets | Yes | We utilize two news datasets in our experiment. The first dataset is a widely used fake news detection dataset collected from a fact-checking website, Gossip Cop (Shu et al. 2020b). Each sample contains the news claim, content, metadata, label, and social engagements. ... The second dataset is the CNN/Daily Mail news highlight dataset (Hermann et al. 2015) which contains the news content and selected highlight. |
| Dataset Splits | Yes | For the dataset splitting, we randomly sample 75% training set, 15% validation set, and 10% test set in the Gossip Cop dataset and follow the same splitting setting in (See, Liu, and Manning 2017). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions software like "Open NMT", "RoBERTa", "spaCy", and "Adam" but does not specify their version numbers. |
| Experiment Setup | Yes | We tune the hyper-parameter λ on the validation set. The encoder of FACTGEN is 4 blocks of SA block with 12 attention heads and 3072 hidden units. The weight of the decoder is initialized with the median pre-trained GPT-2 (Radford et al. 2019) model. The claim reconstruction module is 3 blocks of SA block with 4 attention heads and 256 hidden sizes and Pmask is set as 0.5. The optimizer is Adam (Kingma and Ba 2014) with β1 = 0.9 and β2 = 0.998. It should be noticed that the learning rate for the encoder is 1e 3, for the decoder is 1e 5, and 5e 5 for the claim reconstruction. The number of retrieved documents k1 and sentences k2 is set to 10 and 5 respectively. The epochs1 and epochs2 in the training schedule are set to 4 and 2 respectively. During decoding we used Nucleus Sampling (top-p) with p = 0.9. |