Learning Locality and Isotropy in Dialogue Modeling
Authors: Han Wu, Haochen Tan, Mingjie Zhan, Gangming Zhao, Shaoqing Lu, Ding Liang, Linqi Song
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that our approach significantly outperforms current state-of-the-art models on three open-domain dialogue tasks with eight benchmarks. More in-depth analyses further confirm the effectiveness of our proposed approach. |
| Researcher Affiliation | Collaboration | Han Wu1,2, Haochen Tan1,2, Mingjie Zhan3, Gangming Zhao3, Shaoqing Lu3, Ding Liang3, Linqi Song1,2, 1Department of Computer Science, City University of Hong Kong 2City University of Hong Kong Shenzhen Research Institute 3Sensetime Research |
| Pseudocode | No | The paper describes its methodology using equations but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | We release the code at https://github.com/hahahawu/Sim DRC. |
| Open Datasets | Yes | We evaluate our approach on Daily Dialog (Li et al., 2017) and LCCC (Wang et al., 2020), wherein Daily Dialog is a multi-turn open-domain English dialogue dataset and LCCC is a open-domain Chinese short-text conversation dataset. [...] All datasets used in this work have been publicly released. |
| Dataset Splits | Yes | Du Conv annotations are split into 80%/10%/10% as train/dev/test sets, while News Dialog and Personal Dialog are used as out-of-domain test sets. [...] All hyper-parameters are selected from the development set. |
| Hardware Specification | Yes | The training process on the Daily Dialog and LCCC datasets takes 0.7 hours and 4 hours on four A100 GPUs, respectively. [...] The training process on each dataset takes around 10 hours on a single A100 GPU. [...] The training process on the Du Conv training set takes around 2 hours on two A100 GPUs. |
| Software Dependencies | No | The paper mentions software components like Huggingface Libraries, Adam optimizer, BERTScore, BARTScore, and BLEURT, but it does not specify version numbers for these, which is required for reproducible software dependencies. |
| Experiment Setup | Yes | We use a batch size of 128 and truncate the training samples to a maximum length of 256. The parameters of the models are initialized from Huggingface Libraries (Wolf et al., 2019a) and updated by Adam optimizer (Kingma & Ba, 2015) with a learning rate of 3e-5. The margin values of Sim CTG and Sim DRC are set to 0.5 and 0.7, respectively. The loss weight α is 0.3. |