A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning
Authors: Yuanning Cui, Zequn Sun, Wei Hu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct evaluation on 43 different KGs in both transductive and inductive settings. Results indicate that the proposed KG-ICL outperforms baselines on most datasets, showcasing its outstanding generalization and universal reasoning capabilities. We conduct extensive experiments on 43 datasets to validate the effectiveness of our model. |
| Researcher Affiliation | Academia | Yuanning Cui , Zequn Sun , Wei Hu State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China National Institute of Healthcare Data Science, Nanjing University, Nanjing, China |
| Pseudocode | Yes | We describe our reasoning process in Algorithm 1 of Appendix B. |
| Open Source Code | Yes | The source code is accessible on Git Hub: https://github.com/nju-websoft/KG-ICL. |
| Open Datasets | Yes | We conduct experiments on 43 datasets of various schemata and sizes to evaluate our model. The datasets fall into three groups: (i) 14 inductive datasets, including 12 datasets in Gra IL [1] and 2 datasets in ILPC 2022 [52], (ii) 13 fully-inductive datasets in [5], and (iii) 16 transductive datasets, including FB15k-237 [53], WN18RR [12], NELL-995, [54], YAGO3-10 [55], 3 datasets in Co DEx [56], 5 datasets in [57], Aristo V4 [58], DBpedia100k [59], Concept Net100k [60], and Hetionet [61]. The statistics of datasets are in Appendix F. |
| Dataset Splits | Yes | We use Adam optimizer and set the learning rate to 0.001 and the patience of early stopping to 5. The statistics of datasets are in Appendix F. |
| Hardware Specification | Yes | The pre-training process is conducted on a workstation with two Intel Xeon Gold CPUs, four NVIDIA RTX A6000 GPUs, and Ubuntu 18.04 LTS. |
| Software Dependencies | No | The paper mentions 'Ubuntu 18.04 LTS' as the operating system, but does not specify versions for other software dependencies like programming languages (e.g., Python), libraries (e.g., PyTorch, TensorFlow), or other relevant tools. |
| Experiment Setup | Yes | Under the framework in Section 4, we implement an in-context reasoning model KG-ICL, which employs a 5-shot 3-hop prompt graph as context, along with 3 stacked layers for prompt graph encoding, and 6 stacked layers for KG encoding and reasoning, i.e., M = 5, k = 3, L = 3 and N = 6. The dimension d of the hidden layers is set to 32. ... We use Adam optimizer and set the learning rate to 0.001 and the patience of early stopping to 5. |