KICE: A Knowledge Consolidation and Expansion Framework for Relation Extraction
Authors: Yilin Lu, Xiaoqiang Wang, Haofeng Yang, Siliang Tang
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our framework is verified on relation extraction (RE) task, and the experiments on TACRED show that the model performance (F1) grows from 33.24% to 79.84% with the enrichment of knowledge, outperforming all the baselines including other knowledgeable methods. |
| Researcher Affiliation | Academia | School of Computer Science, Zhejiang University {22121281, xq.wang, 3190105301, siliang}@zju.edu.cn |
| Pseudocode | No | The paper describes the framework's steps in paragraph form and through diagrams, but does not include structured pseudocode or an algorithm block labeled as such. |
| Open Source Code | No | The paper does not contain an explicit statement about releasing the source code for the described methodology, nor does it provide a direct link to a code repository. |
| Open Datasets | Yes | Our experiments are conducted on two benchmark datasets, including TACRED (Zhang et al. 2017) and Re-TACRED (Stoica, Platanios, and P oczos 2021). |
| Dataset Splits | Yes | for each dataset we randomly sample 5% training data as initial seeds Dseed and take the rest as unlabeled data Du. Since the advantage of the large development set is against our label-scarce setting, as suggested in recent work (Gao, Fisch, and Chen 2021), we keep the development set Ddev of the same size as the seeds size, donated as |Ddev| = |Dseed|. Datasets Class Num. Train Dev Test TACRED 41 68124 22631 15485 Re-TACRED 40 58465 19584 13418 |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory, or cloud instance types) used for running the experiments. |
| Software Dependencies | No | The paper mentions using 'BERT (Devlin et al. 2019)' and 'PLMs' but does not specify version numbers for any ancillary software dependencies like programming languages, libraries, or frameworks. |
| Experiment Setup | Yes | Parameters Settings 1) In Rule Generator, the threshold TH is set to 0.97. After feeding the prompt to PLM, nc = 3 words are picked to represent concept patterns and nr = 5 words for relation patterns. 2) If one step is in Self-Reviewing Module, then n R = 120 self-inferred rules are generated. 3) If one step is in Rule-induced Breakthrough Learning Module, it will ask annotation for na = 60 most confusing data. To make KICE more reproducible, the human annotation for each data is the same as its original label in the dataset. In each step, the training set is built by nd = 200 weakly labeled data with the highest annotated confidence. In the model training procedure, the threshold ΞΎ is set to 0.5. For dataset TACRED and Re-TACRED, after the Knowledge Stimulation step, we report the KICE s performance after 4 iterations with Knowledge Consolidation and Knowledge Expansion steps conducted alternatively. |