Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Modeling Source Syntax and Semantics for Neural AMR Parsing
Authors: DongLai Ge, Junhui Li, Muhua Zhu, Shoushan Li
IJCAI 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on an English benchmark dataset show that our two approaches achieve significant improvement of 3.1% and 3.4% F1 scores over a strong seq2seq baseline. |
| Researcher Affiliation | Collaboration | Donglai Ge1 , Junhui Li1 , Muhua Zhu2 and Shoushan Li1 1School of Computer Science and Technology, Soochow University, Suzhou, China 2Alibaba Group, Hangzhou, China |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper refers to 'https://github.com/Rik VN/AMR' for pre-processing and post-processing scripts provided by [van Noord and Bos, 2017], but does not state that the source code for their own proposed methodology is open-source or available. |
| Open Datasets | Yes | For evaluation of our approach, we use the sentences annotated with AMRs from the LDC release LDC2017T10. |
| Dataset Splits | Yes | The dataset consists of 36,521 training AMRs, 1,368 development AMRs and 1,371 testing AMRs. |
| Hardware Specification | Yes | In all experiments, we train the models for 250K steps on a single K40 GPU. |
| Software Dependencies | No | The paper mentions 'Allen NLP' and 'tensor2tensor' as tools used, but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | In parameter setting, we set the number of layers in both the encoder and decoder to 6. For optimization we use Adam with β1 = 0.1 [Kingma and Ba, 2015]. The number of heads is set to 8. In addition, we set the hidden size to 512 and the batch token-size to 4096. In beam searching, we increase the extra length as 100 from default 50. We also set Google NMT length penalty parameter α = 1.0 to encourage longer generation. In all experiments, we train the models for 250K steps on a single K40 GPU. |