CrossNER: Evaluating Cross-Domain Named Entity Recognition
Authors: Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, Pascale Fung13452-13460
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We then conduct comprehensive experiments to explore the effectiveness of leveraging different levels of the domain corpus and pre-training strategies to do domain-adaptive pre-training for the crossdomain task. Results show that focusing on the fractional corpus containing domain-specialized entities and utilizing a more challenging pre-training strategy in domain-adaptive pre-training are beneficial for the NER domain adaptation, and our proposed method can consistently outperform existing cross-domain NER baselines. |
| Researcher Affiliation | Academia | Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, Pascale Fung Center for Artificial Intelligence Research (CAi RE) The Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong zihan.liu@connect.ust.hk, pascale@ece.ust.hk |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code and data are available at https://github.com/zliucr/Cross NER. |
| Open Datasets | Yes | To collect Cross NER, we first construct five unlabeled domain-specific (politics, natural science, music, literature and AI) corpora from Wikipedia. Then, we extract sentences from these corpora for annotating named entities. ... We collect 1000 development and test examples for each domain and a small size of data samples (100 or 200) in the training set for each domain since we consider a low-resource scenario for target domains. |
| Dataset Splits | Yes | We collect 1000 development and test examples for each domain and a small size of data samples (100 or 200) in the training set for each domain since we consider a low-resource scenario for target domains. (Table 1 further specifies: Train #, Dev #, Test # for each domain, e.g., Politics: 200 Train, 541 Dev, 651 Test) |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments, such as GPU models, CPU models, or specific cloud instances. |
| Software Dependencies | No | The paper mentions using BERT (Devlin et al. 2019) but does not specify its version or other software dependencies with version numbers, such as Python or PyTorch versions. |
| Experiment Setup | Yes | We consider the Co NLL2003 English NER dataset (Tjong Kim Sang and De Meulder 2003) from Reuters News... as the source domain and five domains in Cross NER as target domains. Our model is based on BERT (Devlin et al. 2019)... and we follow Devlin et al. (2019) to fine-tune BERT on the NER task. More training details are in the Appendix. Before training on the source or target domains, we conduct the DAPT on BERT when the unlabeled domain-related corpus is leveraged. Moreover, in the DAPT, different types of unlabeled corpora are investigated (i.e., domain-level, entity-level, task-level and integrated corpora), and different masking strategies are inspected (i.e., token-level and span-level masking). |