Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables
Authors: Haisong Gong, Weizhi Xu, Shu Wu, Qiang Liu, Liang Wang
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comprehensive experiments on the large fact checking dataset FEVEROUS demonstrate the effectiveness of Heter FC. ... In this section, we conduct comprehensive experiments to answer the following research questions: ... We utilize the extensive FEVEROUS dataset in our experiments ... Metrics Two metrics gauge the model s performance: the Feverous score and label accuracy. ... Table 1: Comparison of models on Feverous task |
| Researcher Affiliation | Collaboration | Haisong Gong1,2, Weizhi Xu3, Shu Wu1,2*, Qiang Liu1,2, Liang Wang1,2 1Center for Research on Intelligent Perception and Computing State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation, Chinese Academy of Sciences 2School of Artificial Intelligence, University of Chinese Academy of Sciences 3Byte Dance Inc. |
| Pseudocode | No | The paper includes architectural diagrams and mathematical equations but no structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code will be released at: https://github.com/Deno-V/Heter FC. |
| Open Datasets | Yes | Following prior research, we utilize the extensive FEVEROUS dataset in our experiments (Aly et al. 2021). ... The dataset is divided into a training set of 71,291 claims, a development set of 7,890 claims, and a blind test set available on an online judging system2. 2https://eval.ai/web/challenges/challenge-page/1091/overview |
| Dataset Splits | Yes | The dataset is divided into a training set of 71,291 claims, a development set of 7,890 claims, and a blind test set available on an online judging system2. |
| Hardware Specification | Yes | All experiments run on a server with an AMD EPYC 7742 (256) @ 2.250GHz CPU and one NVIDIA A100 GPU. |
| Software Dependencies | No | The paper mentions using a 'RoBERTa model' and the 'Adam optimizer' but does not provide specific version numbers for software libraries or dependencies, such as PyTorch, HuggingFace Transformers, or CUDA. |
| Experiment Setup | Yes | We use the Adam optimizer (Kingma and Ba 2015) with learning rates of 1e-5 for language model parameters and 1e-3 for others, employing a linear scheduler with a 20% warm-up rate. Ro BERTa-Large serves as the PLM, with window size w and R-GCN layer count k set to 2. ... We select β = 1.2 as optimal for Heter FC. |