Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Inference on Syntactic and Semantic Structures for Machine Comprehension
Authors: Chenrui Li, Yuanbin Wu, Man Lan
AAAI 2018 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on the MCTest dataset demonstrate that the proposed models are highly competitive with state-of-the-art machine comprehension systems. |
| Researcher Affiliation | Academia | Chenrui Li,1 Yuanbin Wu,1, 2 Man Lan1, 2 1School of Computer Science and Software Engineering, East China Normal University 2Shanghai Key Laboratory of Multidimensional Information Processing {lcrr2011}@163.com, EMAIL |
| Pseudocode | Yes | Algorithm 1 Message passing for the tree model ... Algorithm 2 Hidden Variable Perceptron |
| Open Source Code | No | The paper does not provide any explicit statement or link regarding the availability of its source code. |
| Open Datasets | Yes | The MCTest dataset (Richardson, Burges, and Renshaw 2013) contains 660 stories written for elementary grade school level students. |
| Dataset Splits | Yes | We use the of๏ฌcial training, development and testing splitting. |
| Hardware Specification | Yes | The testing time is about 0.5s per question on a single CPU with 8G RAM. |
| Software Dependencies | No | The paper mentions 'Stanford Core NLP for coreference resolution', 'Stanford parser for dependency parsing', 'SEMAFOR frame-semantic parser', and 'word2vec' but does not specify version numbers for any of these software dependencies. |
| Experiment Setup | Yes | We tune the algorithm parameter on the development set, and set C = 0.001, M = 10. |