SG-Net: Syntax-Guided Machine Reading Comprehension
Authors: Zhuosheng Zhang, Yuwei Wu, Junru Zhou, Sufeng Duan, Hai Zhao, Rui Wang9636-9643
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on popular benchmarks including SQu AD 2.0 and RACE show that the proposed SG-Net design helps achieve substantial performance improvement over strong baselines. |
| Researcher Affiliation | Academia | 1Department of Computer Science and Engineering, Shanghai Jiao Tong University 2Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China 3Mo E Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China 4College of Zhiyuan, Shanghai Jiao Tong University, China 5National Institute of Information and Communications Technology (NICT), Kyoto, Japan |
| Pseudocode | No | The paper provides architectural diagrams and mathematical formulations but does not include any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/cooelf/SG-Net. |
| Open Datasets | Yes | Our experiments and analysis are carried on two data sets, involving span-based and multi-choice MRC...SQu AD 2.0 (Rajpurkar, Jia, and Liang 2018)...Large-scale Re Ading Comprehension Dataset From Examinations (RACE) dataset (Lai et al. 2017)...Penn Treebank (PTB) (Marcus, Santorini, and Marcinkiewicz 1993) test set |
| Dataset Splits | Yes | We sort the questions from SQu AD dev set according to the length and group them into 20 subsets split by equal range of question length and equal amount of questions...The initial learning rate is set in {8e-6, 1e-5, 2e-5, 3e-5} with warm-up rate of 0.1 and L2 weight decay of 0.01. The batch size is selected in {16, 20, 32}. The maximum number of epochs is set to 3 or 10 depending on tasks. The weight α in the dual context aggregation is 0.5. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for experiments, such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | The paper mentions using 'BERT' and a 'dependency parser from Zhou and Zhao (2019)' but does not specify version numbers for any software components or libraries required for reproducibility. |
| Experiment Setup | Yes | The initial learning rate is set in {8e-6, 1e-5, 2e-5, 3e-5} with warm-up rate of 0.1 and L2 weight decay of 0.01. The batch size is selected in {16, 20, 32}. The maximum number of epochs is set to 3 or 10 depending on tasks. The weight α in the dual context aggregation is 0.5. All the texts are tokenized using wordpieces, and the maximum input length is set to 384 for both of SQu AD and RACE. |