A Deep Cascade Model for Multi-Document Reading Comprehension
Authors: Ming Yan, Jiangnan Xia, Chen Wu, Bin Bi, Zhongzhou Zhao, Ji Zhang, Luo Si, Rui Wang, Wei Wang, Haiqing Chen7354-7361
AAAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiment results show that the proposed method outperforms the previous state-of-the-art methods on two large-scale multi-document benchmark datasets, i.e., Trivia QA and Du Reader. In addition, our online system can stably serve typical scenarios with millions of daily requests in less than 50ms.This section presents the experimental methodology. We first verify the effectiveness of our model on two benchmark datasets: Trivia QA (Joshi et al. 2017) and Du Reader (He et al. 2017). Then we test our model in operational online environment, which can stably and effectively serve different scenarios promptly. |
| Researcher Affiliation | Industry | Ming Yan, Jiangnan Xia, Chen Wu, Bin Bi, Zhongzhou Zhao, Ji Zhang, Luo Si, Rui Wang, Wei Wang, Haiqing Chen Alibaba Group {ym119608, jiangnan.xjn, wuchen.wc, b,bi}@alibaba-inc.com {zhongzhou.zhaozz, zj122146, luo.si, masi.wr, hebian.ww, haiqing.chenhq}@alibaba-inc.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. Figure 1 and 2 are diagrams illustrating the model architecture. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described. It does not mention releasing code or provide any repository links. |
| Open Datasets | Yes | We choose the Trivia QA Web and Du Reader benchmark datasets to test our method, since both of them are multi-document MRC datasets which is more realistic and challenging. Trivia QA is a recently released large-scale multi-document MRC datasets... Du Reader is so far the largest Chinese MRC dataset... |
| Dataset Splits | Yes | We choose K = 4 and N = 2 for the good performance when evaluating on the dev set. We conduct an in-depth ablation study on the development set of Du Reader and Trivia QA, which is shown in Table 3. |
| Hardware Specification | Yes | All models are trained on Nvidia Tesla M40 GPU with Cudnn LSTM cell in Tensorflow 1.3. |
| Software Dependencies | Yes | All models are trained on Nvidia Tesla M40 GPU with Cudnn LSTM cell in Tensorflow 1.3. |
| Experiment Setup | Yes | For the multi-task deep attention framework, we adopt the Adam optimizer for training, with a mini-batch size of 32 and initial learning rate of 0.0005. The hidden size of LSTM is set as 150 for Trivia QA and 128 for Du Reader. The task-specific hyper-parameters λ1 and λ2 in Equ. 15 are set as λ1 = λ2 = 0.5. Regularization parameter δ in Equ. 16 is set as a small value of 0.01. |