Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
OneRel: Joint Entity and Relation Extraction with One Module in One Step
Authors: Yu-Ming Shang, Heyan Huang, Xianling Mao11285-11293
AAAI 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experimental results on two widely used datasets demonstrate that the proposed method performs better than the state-of-the-art baselines, and delivers consistent performance gain on complex scenarios of various overlapping patterns and multiple triples. |
| Researcher Affiliation | Academia | Yu-Ming Shang1, Heyan Huang1,2, Xian-Ling Mao1* 1School of Computer Science & Technology, Beijing Institute of Technology, Beijing, China 2Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing, China EMAIL |
| Pseudocode | No | The paper describes the methodology and algorithms in prose and mathematical formulas, but it does not include a clearly labeled pseudocode block or algorithm section. |
| Open Source Code | No | The paper does not contain any explicit statement about open-sourcing the code for the described methodology, nor does it provide a link to a code repository. |
| Open Datasets | Yes | Following previous works (Wei et al. 2020; Wang et al. 2020; Zheng et al. 2021), we evaluate our model and all baselines on two widely used datasets: NYT (Riedel, Yao, and Mc Callum 2010) and Web NLG (Gardent et al. 2017). |
| Dataset Splits | Yes | Detailed statistics of the two datasets are described in Table 1. Train Valid Test Relations Normal SEO EPO HTO N=1 N=2 N=3 N=4 N>5 Triples NYT 56,195 4,999 5,000 24... Web NLG 5,019 500 703 171... |
| Hardware Specification | Yes | In our experiments, all training process is completed on a work station with an AMD 7742 2.25G CPU, 256G memory, a single RTX 3090 GPU, and Ubuntu 20.04. |
| Software Dependencies | No | The paper mentions 'pre-trained BERT', 'Huggingface' for the BERT model, and 'Adam algorithm' for optimization, but it does not specify software dependencies with version numbers (e.g., Python 3.x, PyTorch x.x, TensorFlow x.x). |
| Experiment Setup | Yes | Specifically, the batch size is set to 8/6 on NYT/Web NLG, and all parameters are optimized by Adam algorithm (Kingma and Ba 2015) with a learning rate of 1e-5. The dimension of the hidden layer de is set to 3 d, the dropout probability in equation (4) is 0.1, the max sequence length is set to 100. |