MKGL: Mastery of a Three-Word Language
Authors: Lingbing Guo, Zhongpu Bo, Zhuo Chen, Yichi Zhang, Jiaoyan Chen, Lan Yarong, Mengshu Sun, Zhiqiang Zhang, Yangyifei Luo, Qian Li, Qiang Zhang, Wen Zhang, Huajun Chen
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we evaluate the performance of the proposed MKGL through extensive experiments, comparing it against both LLM-based and KG embedding methods. The source code and datasets are available at github.com/zjukg/MKGL. |
| Researcher Affiliation | Collaboration | Lingbing Guo1,2, Zhongpu Bo3, Zhuo Chen1,2, Yichi Zhang1,2, Jiaoyan Chen4, Yarong Lan1,2, Mengshu Sun3, Zhiqiang Zhang3, Yangyifei Luo5, Qian Li6, Qiang Zhang1,2, Wen Zhang7,2 and Huajun Chen1,2 1College of Computer Science and Technology, Zhejiang University 2ZJU-Ant Group Joint Lab of Knowledge Graph 3Ant Group 4Department of Computer Science, The University of Manchester 5School of Computer Science and Engineering, Beihang University 6School of Computer Science, Beijing University of Posts and Telecommunications 7School of Software Technology, Zhejiang University |
| Pseudocode | Yes | Algorithm 1 MKGL for KG Completion |
| Open Source Code | Yes | The source code and datasets are available at github.com/zjukg/MKGL. |
| Open Datasets | Yes | We evaluate MKGL on the FB15k-237 and WN18RR datasets, which are widely used by most KG completion methods [22, 23, 26, 28, 34, 56, 57]. We also evaluate MKGL on the inductive version of these two datasets [58]. |
| Dataset Splits | Yes | Table 6: Dataset statistics. Dataset # Relation Train Valid Test # Entity # Triplet # Entity # Evaluation # Fact FB15k-237 237 14,541 272,115 17,535 20,466 WN18RR 11 40,943 86,835 3,034 3,134 |
| Hardware Specification | Yes | For our experiments, we employ Llama-2-7b [11] as the base LLM and train MKGL using 8 A100 GPUs. |
| Software Dependencies | No | The paper mentions using "Llama-2-7b [11] as the base LLM" which is a specific model version. However, it does not provide specific version numbers for other key software components, libraries, or programming languages (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | Full hyper-parameter details are available in Appendix D. We evaluate performance using MRR (mean reciprocal rank of target entities) and Hits@k (percentage of target entities ranked in the top k). Table 5: Hyper-parameter settings in the main experiments. Datasets LLM Lo RA r Lo RA dropout Lo RA target modules train batch size per device loss criterion gradient accumulation steps optimizer |