DCMN+: Dual Co-Matching Network for Multi-Choice Reading Comprehension
Authors: Shuailiang Zhang, Hai Zhao, Yuwei Wu, Zhuosheng Zhang, Xi Zhou, Xiang Zhou9563-9570
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | DCMN equipped with the two strategies (DCMN+) obtains state-of-the-art results on five multi-choice reading comprehension datasets from different domains: RACE, Sem Eval-2018 Task 11, ROCStories, COIN, MCTest. [...] Table 4 reports the experimental results on RACE and its two subtasks: RACE-M and RACE-H. |
| Researcher Affiliation | Collaboration | Shuailiang Zhang,1,2,3 Hai Zhao,1,2,3 Yuwei Wu,1,2,3 Zhuosheng Zhang,1,2,3 Xi Zhou,4 Xiang Zhou4 1Department of Computer Science and Engineering, Shanghai Jiao Tong University 2Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China 3Mo E Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China 4Cloud Walk Technology, Shanghai, China |
| Pseudocode | No | The paper describes its model architecture and components but does not include explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | 1Our code is at https://github.com/Qzsl123/dcmn. |
| Open Datasets | Yes | We evaluate our model on five multi-choice MRC datasets from different domains. Statistics of these datasets are detailed in Table 3. [...] RACE (Lai et al. 2017): [...] Sem Eval-2018 Task11 (Ostermann et al. 2018): [...] ROCStories (Mostafazadeh et al. 2016): [...] MCTest (Richardson, Burges, and Renshaw 2013): [...] COIN Task 1 (Ostermann et al. 2018): |
| Dataset Splits | Yes | In Table 5, we focus on the contribution of main components (DCMN, passage sentence selection and answer option interaction) in our model. (Table 5 title: Ablation study on RACE dev set.) [...] Figure 2 shows the results of passage sentence selection (Pss) on COIN and RACE dev set with different numbers of selected sentences (Top K). |
| Hardware Specification | Yes | We train for 10 epochs with batch size 8 using eight 1080Ti GPUs when BERTlarge and XLNetlarge are used as the encoder. |
| Software Dependencies | No | The paper mentions 'Bert Adam (Devlin et al. 2019) optimizer' but does not specify version numbers for any software components, libraries, or programming languages used. |
| Experiment Setup | Yes | In our experiments, the max input sequence length is set to 512. A dropout rate of 0.1 is applied to every BERT layer. We optimize the model using Bert Adam (Devlin et al. 2019) optimizer with a learning rate 2e-5. We train for 10 epochs with batch size 8 using eight 1080Ti GPUs when BERTlarge and XLNetlarge are used as the encoder. Batch size is set to 16 when using BERTbase and XLNetbase as the encoder1. |