Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning
Authors: Xiang Chen, Lei Li, Ningyu Zhang, Xiaozhuan Liang, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, Huajun Chen
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that RETROPROMPT can obtain better performance in both few-shot and zero-shot settings. |
| Researcher Affiliation | Collaboration | Xiang Chen1,2 , Lei Li1,2 , Ningyu Zhang1,2 , Xiaozhuan Liang1,2, Shumin Deng1,2, Chuanqi Tan3, Fei Huang3, Luo Si3, Huajun Chen1,2 1Zhejiang University & AZFT Joint Lab for Knowledge Engine, China 2Hangzhou Innovation Center, Zhejiang University, China 3Alibaba Group, China |
| Pseudocode | No | No explicit pseudocode or algorithm blocks found. |
| Open Source Code | Yes | Code is available in https://github.com/zjunlp/Prompt KG/tree/main/research/ Retro Prompt. |
| Open Datasets | Yes | We evaluate RETROPROMPT on several types of natural language understanding tasks, including single sentence classification tasks (SST-2 [54], MR [43], and CR [19]) and sentence pair classification tasks (MNLI [58], QNLI [46], and QQP5). To further evaluate the effectiveness of the proposed approach with multi-class classification, we also conduct experiments on the information extraction tasks, including Few NERD [8], Sem Eval 2010 Task 8 (Sem Eval) [17], and TACRED [61]. |
| Dataset Splits | No | We follow the few-shot setting of LM-BFF [12] to conduct 4-shot and 16-shot experiments and evaluate the average performance with a fixed set of seeds, Sseed, across several different sampled Dtrain for each task. No explicit train/validation/test dataset splits (percentages or counts) are provided in the main text for the overall datasets beyond the few-shot setting description. |
| Hardware Specification | Yes | The experiments are implemented on 1 NVIDIA V100 and utilize Pytorch [44] as the base library. |
| Software Dependencies | No | The paper mentions software like Pytorch [44], Ro BERTalarge [38], and FAISS [21], but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | We list the specific experimental settings and tuning retrieve parameters in Appendix C and D. |