Iterative Utterance Segmentation for Neural Semantic Parsing
Authors: Yinuo Guo, Zeqi Lin, Jian-Guang Lou, Dongmei Zhang12937-12945
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on GEO, COMPLEXWEBQUESTIONS and FORMULAS show that our framework can consistently improve performances of neural semantic parsers in different domains. On data splits that require compositional generalization, our framework brings significant accuracy gains: GEO 63.1 81.2, FORMULAS 59.7 72.7, COMPLEXWEBQUESTIONS 27.1 56.3. |
| Researcher Affiliation | Collaboration | Yinuo Guo1 , Zeqi Lin2, Jian-Guang Lou2, Dongmei Zhang2 1Key Laboratory of Computational Linguistics, School of EECS, Peking University 2Microsoft Research Asia gyn0806@pku.edu.cn, {Zeqi.Lin, jlou, dongmeiz}@microsoft.com |
| Pseudocode | No | The paper describes the steps of the iterative process and the components of the framework in descriptive text, but it does not provide a formal pseudocode block or algorithm. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing its own source code or a link to a code repository for its methodology. |
| Open Datasets | Yes | We conduct experiments on three datasets: GEO (Zelle and Mooney 1996), COMPLEXWEBQUESTIONS (Talmor and Berant 2018) and FORMULAS (a new dataset). Examples of (utterance, meaning representation) instances in these three datasets are shown in Table 1. |
| Dataset Splits | Yes | COMPLEXWEBQUESTIONS: There are 27,734/3,480/3,475 train/dev/test examples in this dataset. FORMULAS: We randomly split this dataset into three parts: 800 instances for training, 268 instances for development, and 268 instances for test. GEO: GEO can be splitted into train/test sets in two different ways: (1) Standard split (Zettlemoyer and Collins 2005): this split ensures that no natural language question is repeated between the train and test sets; (2) Compositional split (Finegan-Dollak et al. 2018): this split ensures that neither questions nor meaning representations(anonymizing named entities) are repeated . |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models used for running the experiments. |
| Software Dependencies | No | The paper mentions 'Adam optimizer with default settings (in PYTORCH)' but does not specify the version of PyTorch or any other software dependencies with their respective version numbers. |
| Experiment Setup | Yes | We set the dimension of word embedding to 300. In SEQ2SEQ/SEQ2TREE base parsers, we set the dimension of hidden vector to 512. In the utterance segmentation model, the dimension of hidden vector is set to 300. We use the Adam optimizer with default settings (in PYTORCH) and a dropout layer with the rate of 0.5. The training process lasts 100 epochs with batch size 64. For the TRANSFORMER base parser in COMPLEXWEBQUESTIONS, we follow the settings in Vaswani et al. (2017). |