Fact Checking via Evidence Patterns
Authors: Valeria Fionda, Giuseppe Pirrò
IJCAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We implemented our algorithms in the CHEEP (CHeck via Evidenc E Ptterns) system using: (i) Java with the KG loaded in memory (CHEEP-M); and, (ii) by accessing the KG and verifying patterns via SPARQL (CHEEP-Q). ... Experiments have been performed on a Mac Book Pro 2.6 GHz Intel Core i5 with 8Gbs memory. To test the performance, we used the area under the Receiver Operating Characteristic Curve (AUC) as also done in previous work [Shiralkar et al., 2017]. ... Table 1 summarizes the results in terms of average AUC (real world datasets are reported on the left). |
| Researcher Affiliation | Academia | Valeria Fionda1, Giuseppe Pirr o2, 1 De Ma CS, University of Calabria, Rende, Italy 2 ICAR-CNR, Rende, Italy fionda@mat.unical.it, pirro@icar.cnr.it |
| Pseudocode | Yes | Algorithm 1: find Evidence Patterns and Algorithm 2: build Evidence Graph are present in the paper. |
| Open Source Code | Yes | The implementations are available online1. 1https://factcheckingkgs.wordpress.com |
| Open Datasets | Yes | We considered the DBpedia dataset used by [Shiralkar et al., 2017] (24M triples, and 663 predicates). ... For the evaluation we considered 10 datasets used in Shiralkar et al. [Shiralkar et al., 2017] and for which the ground truth is available [KStream, 2018]. |
| Dataset Splits | No | The paper mentions using 10 datasets from Shiralkar et al. [Shiralkar et al., 2017] and KStream [KStream, 2018], but it does not specify any explicit train/validation/test dataset splits, percentages, or sample counts within its text. |
| Hardware Specification | Yes | Experiments have been performed on a Mac Book Pro 2.6 GHz Intel Core i5 with 8Gbs memory. |
| Software Dependencies | No | The paper states 'We implemented our algorithms in the CHEEP... using: (i) Java... and, (ii) by accessing the KG and verifying patterns via SPARQL (CHEEP-Q)', but it does not provide specific version numbers for Java or any other software libraries or dependencies. |
| Experiment Setup | Yes | We used the evidence score in eq. (9) and d =3 in Algorithm 1. As for CHEEP, the notation CHEEP(#Predicates,#Patterns) represents a configuration for a given number of predicates related to p and evidence patterns to be verified. We considered CHEEP(10,5), and CHEEP(10,10). |