Language Models as Knowledge Embeddings
Authors: Xintao Wang, Qianyu He, Jiaqing Liang, Yanghua Xiao
IJCAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that LMKE achieves state-of-the-art performance on KE benchmarks of link prediction and triple classification, especially for long-tail entities. We conduct extensive experiments on widely-used KE benchmarks |
| Researcher Affiliation | Academia | 1Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University 2Fudan-Aishu Cognitive Intelligence Joint Research Center |
| Pseudocode | No | No pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | Yes | Our codes are available at https://github.com/Neph0s/LMKE |
| Open Datasets | Yes | We experiment on four popular benchmark datasets: FB13 [Socher et al., 2013], FB15k237 [Toutanova, 2015], UMLS [Dettmers et al., 2018] and WN18RR [Dettmers et al., 2018], whose statistics are shown in Table 3. |
| Dataset Splits | Yes | We experiment on four popular benchmark datasets: FB13 [Socher et al., 2013], FB15k237 [Toutanova, 2015], UMLS [Dettmers et al., 2018] and WN18RR [Dettmers et al., 2018], whose statistics are shown in Table 3. Table 3: Statistics of the datasets. Dataset #Entity #Relation #Train #Dev #Test Avg DL. We search these hyperparameters with BERT-tiny: learning rate of PLMs among {10 4, 5 10 5, 10 5}, learning rate of other components among {10 3, 5 10 4, 10 4, 10 5}, batch size among {12, 16, 32, 64} based on best Hits@10 on the dev set. |
| Hardware Specification | No | No specific hardware details (e.g., GPU model, CPU type, memory) were found. The paper mentions models are "fine-tuned" but does not specify the computational resources used. |
| Software Dependencies | No | The paper mentions using "BERT-base [Devlin et al., 2018] and BERT-tiny [Turc et al., 2019] as the language model" and "Adam as the optimizer." However, specific version numbers for these or other software dependencies (e.g., Python, PyTorch, TensorFlow, CUDA) are not provided. |
| Experiment Setup | Yes | We search these hyperparameters with BERT-tiny: learning rate of PLMs among {10 4, 5 10 5, 10 5}, learning rate of other components among {10 3, 5 10 4, 10 4, 10 5}, batch size among {12, 16, 32, 64} based on best Hits@10 on the dev set. With BERT-base, we set the batch size as 16 for triple classification and 12 for link prediction, which considers both results of BERT-tiny and limited memory. For triple classification, we sample 1 negative triple for each positive triple. |