Making LLMs as Fine-Grained Relation Extraction Data Augmentor
Authors: Yifan Zheng, Wenjun Ke, Qi Liu, Yuting Yang, Ruizhuo Zhao, Dacheng Feng, Jianwei Zhang, Zhi Fang
IJCAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results from the evaluation of Sem Eval, TACRED, and TACREV datasets unequivocally demonstrate that Consist RE outperforms other baselines in F1 values by 1.76%, 3.92%, and 2.53%, respectively, particularly when operating under low-resource experimental conditions. |
| Researcher Affiliation | Academia | 1School of Computer Science and Engineering, Southeast University 2Beijing Institute of Computer Technology and Application 3Key Laboratory of New Generation Artifcial Intelligence Technology and Its Interdisciplinary Applications (Southeast University) 4Laboratory for Big Data and Decision National University of Defense Technology |
| Pseudocode | No | The paper describes the approach in text and with diagrams but does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper references third-party tools (e.g., langchain, FAISS, Stanford Parse) and a backbone model (gpt-3.5-turbo) with their general links, but it does not provide an explicit statement or link to the authors' own source code for Consist RE. |
| Open Datasets | Yes | We conduct our experiments on three public RE datasets: Sem Eval 2010 Task 8 (Sem Eval) [Hendrickx et al., 2009], the TAC Relation Extraction Dataset (TACRED) [Zhang et al., 2017], and the revisited TAC Relation Extraction Dataset (TACREV) [Alt et al., 2020]. |
| Dataset Splits | Yes | The statistics of datasets are presented in Table 1. Dataset #Rel #Train #Val #Test... |
| Hardware Specification | No | The paper does not explicitly describe the hardware used for running experiments (e.g., specific GPU/CPU models, memory specifications). |
| Software Dependencies | No | The paper mentions 'gpt-3.5-turbo' as a backbone model and other tools like 'langchain', 'FAISS', and 'Stanford Parse', but it does not specify version numbers for any of these software dependencies or libraries. |
| Experiment Setup | No | The paper describes aspects of the experimental setup such as sampling low-resource scenarios and augmenting data 3x, but it does not provide specific hyperparameters like learning rate, batch size, number of epochs, or optimizer settings. |