Synergistic Anchored Contrastive Pre-training for Few-Shot Relation Extraction
Authors: Da Luo, Yanglei Gan, Rui Hou, Run Lin, Qiao Liu, Yuxiang Cai, Wannian Gao
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results demonstrate that our framework achieves significant performance enhancements compared to baseline models in downstream FSRE tasks. Furthermore, our approach exhibits superior adaptability to handle the challenges of domain shift and zero-shot relation extraction. Our code is available online at https://github.com/ AONE-NLP/FSRE-Sa Con. |
| Researcher Affiliation | Academia | Da Luo, Yanglei Gan, Rui Hou , Run Lin , Qiao Liu *, Yuxiang Cai , Wannian Gao University of Electronic Science and Technology of China {luoda, yangleigan, hour, runlin, yuxiangcai, wanniangao}@std.uestc.edu.cn, qliu@uestc.edu.cn |
| Pseudocode | No | The paper describes its approach and models using text and mathematical equations but does not provide any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available online at https://github.com/ AONE-NLP/FSRE-Sa Con. |
| Open Datasets | Yes | The pre-training dataset (Peng et al. 2020) is constructed by using Wikipedia articles as the corpus and Wikidata (Vrandeˇci c and Kr otzsch 2014) as the knowledge graph... The fine-tuning process is conducted on Few Rel 1.0 (Han et al. 2018) and Few Rel 2.0 (Gao et al. 2019b) datasets. |
| Dataset Splits | Yes | Our experiments follow the splits used in official benchmarks, which split the dataset into 64 base classes for training, 16 classes for validation, and 20 novel classes for testing. |
| Hardware Specification | Yes | Our Sa Con is trained on the NVIDIA A100 Tensor Core GPU. |
| Software Dependencies | No | It undergoes pre-training on the BERT-base model from the Huggingface Transformer library 1 and uses Adam W (Loshchilov and Hutter 2018) for optimization with the learning rate as 3e-5. [Footnote 1: https://github.com/huggingface/transformers]. While the library is mentioned, a specific version number for the Huggingface Transformer library is not provided. |
| Experiment Setup | Yes | The temperature parameter τ is learnable and initialized to 0.07 from (Wu et al. 2018) and clipped to prevent scaling the logits by more than 100. The epoch and batch size are set to 20 and 128. As for finetuning, we select the same values as those reported in the baseline methods. The batch size and training iteration are set to 4 and 10,000 with learning rate as {1e-5, 2e-5, 1e-6}. |