Dual Task Framework for Improving Persona-Grounded Dialogue Dataset
Authors: Minju Kim, Beong-woo Kwak, Youngwook Kim, Hong-in Lee, Seung-won Hwang, Jinyoung Yeo10912-10920
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on Persona-Chat show that our approach outperforms pretrained LMs by an 11.7 point gain in terms of accuracy. Our extensive experiments validate that, along with linked personas, the response accuracy significantly increases by 11.7% point on Persona-Chat compared to that of using the raw dataset. |
| Researcher Affiliation | Academia | 1Yonsei University 2Seoul National University |
| Pseudocode | Yes | Algorithm 1: Primal-Dual Task Framework |
| Open Source Code | No | The paper does not provide any concrete access to source code for the described methodology. |
| Open Datasets | Yes | Persona-Chat (Zhang et al. 2018), where crowdworkers role-play following the given description of personas to populate dialogues. |
| Dataset Splits | No | The paper mentions using 'development/test sets' of Persona-Chat but does not provide specific percentages or counts for training, validation, or test splits. It refers to the 'evaluation set' but does not specify its size or composition beyond being used for choosing 19 random utterances. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions models like RoBERTa, GPT2, BERT, Bi-encoder, and Cross-encoder, and libraries like BM25, K-NRM, and Conv-KNRM, but does not specify version numbers for these software components or programming languages used. |
| Experiment Setup | Yes | In our setting, we adopt the response selection task (Humeau et al. 2020). As response space U, the model has to pick the correct response from a set of 20 choices, where the remaining 19 were randomly chosen utterances from the evaluation set. To compute the linking score P(p|u; θ) and the cross-entropy loss L, based on the Bi-encoder architecture allowing for fast and real-time inference, we follow the optimization procedure (Logeswaran et al. 2019; Humeau et al. 2020; Jeong et al. 2021) of retrieval-based models (e.g., information retrieval, entity linking, and response selection): The network is trained to maximize the score of the correct persona p with respect to the (randomly sampled) personas of the same batch (i.e., in-batch negatives). + λ L(P( p| u; θLink), P( p| u; θ)) where λ is a preference weight with the distillation loss. |