Neighborhood Intervention Consistency: Measuring Confidence for Knowledge Graph Link Prediction
Authors: Kai Wang, Yu Liu, Quan Z. Sheng
IJCAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results on ten popular KGE models show that our NIC method can effectively estimate the confidence score of each predicted triple. |
| Researcher Affiliation | Academia | 1School of Software Technology, Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian University of Technology, Dalian, Liaoning, 116620, P.R. China 2Department of Computing, Macquarie University, Sydney, NSW 2109, Australia |
| Pseudocode | No | The paper describes the NIC framework with steps and equations, but it does not include a formal pseudocode block or algorithm. |
| Open Source Code | No | The paper does not provide any explicit statement about open-sourcing the code for the methodology or a link to a code repository. |
| Open Datasets | Yes | Our experimental studies are conducted on two widely used datasets, WN18RR [Bordes et al., 2014] and FB15k237 [Toutanova and Chen, 2015]. |
| Dataset Splits | Yes | Table 2: Statistics of the datasets. Dataset |R| |E| #Train #Valid #Test FB15k237 237 14, 541 272, 115 17, 535 20, 466 WN18RR 11 40, 943 86, 845 3, 034 3, 134 |
| Hardware Specification | Yes | All experiments are performed on Intel Core i77700K CPU @ 4.20GHz and NVIDIA Ge Force GTX1080 Ti GPU, and implemented in Python using the Py Torch framework. |
| Software Dependencies | No | The paper states that experiments were 'implemented in Python using the Py Torch framework', but it does not specify version numbers for these software components. |
| Experiment Setup | Yes | For the six high-dimensional KGE models, such as Trans E and Tuck ER, we set their embedding dimensions as 200, while the four low-dimensional models embedding dimension is 32. We select the hyper-parameters in the NIC framework via grid search. Specifically, we empirically select the number of remeasured entities K among {3, 5, 10, 100} and the position number J for computing sequence consistency among {1, 3, 5, 10}. |