Differentiable Reasoning on Large Knowledge Bases and Natural Language
Authors: Pasquale Minervini, Matko Bošnjak, Tim Rocktäschel, Sebastian Riedel, Edward Grefenstette5182-5190
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We report the results of experiments on benchmark datasets Countries (Bouchard, Singh, and Trouillon 2015), Nations, UMLS, and Kinship (Kemp et al. 2006)... Furthermore, since GNTPs allows to experiment on significantly larger datasets, we also report results on the WN18 (Bordes et al. 2013), WN18RR (Dettmers et al. 2018) and FB122 (Guo et al. 2016) datasets. Results are reported in terms of the Area Under the Precision-Recall Curve (AUC-PR) (Davis and Goadrich 2006), Mean Reciprocal Rank (MRR), and HITS@m (Bordes et al. 2013). |
| Researcher Affiliation | Collaboration | 1UCL Centre for Artificial Intelligence, University College London 2Facebook AI Research |
| Pseudocode | No | The paper describes the approach using text and equations but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | 1The Code and supplementary material are available online at https://github.com/uclnlp/gntp. |
| Open Datasets | Yes | We report the results of experiments on benchmark datasets Countries (Bouchard, Singh, and Trouillon 2015), Nations, UMLS, and Kinship (Kemp et al. 2006)... Furthermore, since GNTPs allows to experiment on significantly larger datasets, we also report results on the WN18 (Bordes et al. 2013), WN18RR (Dettmers et al. 2018) and FB122 (Guo et al. 2016) datasets. |
| Dataset Splits | Yes | Table 3 shows an excerpt of validation triples together with their GNTP proof scores and associated proof paths for WN18. AND Datasets and hyperparameters are described in the Appendix. 6 |
| Hardware Specification | No | Figure 2: Number of seconds per epoch required for training on the WN18 dataset using batches of 1000 examples on a GPU. |
| Software Dependencies | No | The paper does not provide specific software names with version numbers for reproducibility. |
| Experiment Setup | No | Datasets and hyperparameters are described in the Appendix. 6 |