R3: Reinforced Ranker-Reader for Open-Domain Question Answering
Authors: Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, Gerry Tesauro, Bowen Zhou, Jing Jiang
AAAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We report extensive experimental results showing that our method significantly improves on the state of the art for multiple open-domain QA datasets. and We evaluate our model on five different datasets and achieve state-of-the-art results on four of the them. |
| Researcher Affiliation | Collaboration | Shuohang Wang,1 Mo Yu,2 Xiaoxiao Guo,2 Zhiguo Wang,2 Tim Klinger,2 Wei Zhang,2 Shiyu Chang,2 Gerald Tesauro,2 Bowen Zhou,3 Jing Jiang1 1School of Information System, Singapore Management University 2AI Foundations Learning, IBM Research AI. Yorktown Heights NY, USA 3JD.COM. Beijing, China |
| Pseudocode | Yes | Algorithm 1 Reinforced Ranker-Reader (R3) |
| Open Source Code | Yes | Code: https://github.com/shuohangwang/mprc. |
| Open Datasets | Yes | We experiment with five different datasets whose statistics are shown in Table 2. Quasar-T... SQu AD... Wiki Movies... Curated TREC... Web Question... For these four datasets under the open-domain QA setting... we build a similar sentence-level Search Index based on English Wikipedia, following Chen et al. 2017a s work. We use the 2016-12-21 dump of English Wikipedia as our sole knowledge source, and build an inverted index with Lucene. and SQu AD (Rajpurkar et al. 2016) |
| Dataset Splits | No | The paper mentions training and testing datasets, but does not explicitly provide details about a validation dataset split (e.g., specific percentages or sample counts for validation). |
| Hardware Specification | No | The paper does not provide specific details on the hardware used for experiments (e.g., GPU/CPU models, memory). |
| Software Dependencies | No | The paper mentions software components like Lucene and GloVe but does not provide specific version numbers for them or any other software dependencies. |
| Experiment Setup | Yes | The number of LSTM layers in Eqn.(4) is set to 3 for the Reader and 1 for the Ranker. Our model is optimized using Adamax (Kingma and Ba 2015). We use fixed Glo Ve (Pennington, Socher, and Manning 2014) word embeddings. We set l to 300, batch size to 30, learning rate to 0.002 and tune the dropout probability. |