Fast and Continual Knowledge Graph Embedding via Incremental LoRA
Authors: Jiajun Liu, Wenjun Ke, Peng Wang, Jiahao Wang, Jinhua Gao, Ziyu Shang, Guozheng Li, Zijie Xu, Ke Ji, Yining Li
IJCAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct experiments on four public datasets and two new datasets with a larger initial scale. Experimental results demonstrate that Fast KGE can reduce training time by 34%-49% while still achieving competitive link prediction performance against state-of-the-art models on four public datasets (average MRR score of 21.0% vs. 21.1%). |
| Researcher Affiliation | Academia | 1School of Computer Science and Engineering, Southeast University 2Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education 3Institute of Computing Technology, Chinese Academy of Sciences 4College of Software Engineering, Southeast University {jiajliu, kewenjun, pwang, wang jh, ziyus1999, gzli, zijiexu, keji, liyining}@seu.edu.cn, {gaojinhua}@ict.ac.cn |
| Pseudocode | No | The paper describes methods and processes (e.g., BFS algorithm) but does not include a formally labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | Yes | The datasets are available at https://github.com/seukgcode/Fast KGE. |
| Open Datasets | Yes | We use six datasets in the main experiments: ENTITY, RELATION, FACT, HYBRID, FB-CKGE, and WN-CKGE. ENTITY, RELATION, FACT, and HYBRID are traditional datasets for CKGE [Cui et al., 2023]... We construct two new datasets for CKGE: FB-CKGE and WNCKGE, which are based on two widely-used KGE datasets FB15K-237 [Dettmers et al., 2018] and WN18RR [Toutanova et al., 2015]. The datasets are available at https://github.com/seukgcode/Fast KGE. |
| Dataset Splits | Yes | The ratio of train, valid, and test sets for all datasets is 3:1:1. |
| Hardware Specification | Yes | All experiments are conducted on the NVIDIA RTX 3090Ti GPU. |
| Software Dependencies | No | The codes of the experiments are supported by PyTorch [Paszke et al., 2019]. (No version numbers provided for PyTorch or other libraries) |
| Experiment Setup | Yes | We choose the batch size from [258, 512, 1024] and the learning rate from [1e-1, 2e-1, 3e-1]. We choose the Adam as the optimizer. In our experiments, we set the entity base rank of Lo RA from the range [10, 50, 100, 150, 200] and the relation base rank to 20. Also, we set the number of Lo RA layers N from the range [2, 5, 10, 20]. We set the embedding size for all methods to 200. To fairness, all the results are from the average of random five running times, and the patience of early stopping is 3 for all methods to compare time efficiency. |