Parse Tree Fragmentation of Ungrammatical Sentences
Authors: Homa B. Hashemi, Rebecca Hwa
IJCAI 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that the proposed fragmentation strategies are competitive with existing methods for making fluency judgments; they also suggest that the overall framework is a promising way to handle syntactically unusual sentences. |
| Researcher Affiliation | Academia | Homa B. Hashemi, Rebecca Hwa Intelligent Systems Program, Computer Science Department University of Pittsburgh hashemi@cs.pitt.edu, hwa@cs.pitt.edu |
| Pseudocode | No | The paper describes algorithms in prose but does not include any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statement about releasing open-source code for the described methodology, nor does it include a link to a code repository. |
| Open Datasets | Yes | English as a Second Language corpus (ESL) The First Certificate in English (FCE) dataset [Yannakoudakis et al., 2011] is a learner s corpus that contains ungrammatical sentences and their corresponding error corrections. Machine Translation corpus (MT) The LIG corpus [Potet et al., 2012] contains 10,881 French-English machine translation outputs and their human post-editions. |
| Dataset Splits | Yes | From this corpus, we create two datasets for the experiments. First, we randomly select 5000 sentences with at least one error; this dataset is for training the CLF fragmentation method as well as for the intrinsic evaluation of different fragmentation methods. Then, we create a second, non-overlapping dataset for the extrinsic evaluation. It consists of 7000 sentences and is representative of the corpus s error distribution; there are 2895 sentences with no error; 2103 with one error; 1092 with two errors; and 910 with 3+ errors. We then build a dataset of 4000 sentences with HTER score more than 0.1 for training CLF and intrinsic evaluation, and a dataset of 6000 sentences with real distribution of HTER scores for extrinsic evaluation. We performed a 10-fold cross validation for the two domains: ESL and MT. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | Yes | The Stanford parser version 3.2.0 [Klein and Manning, 2003] is used to generate parses for all sentences. For all binary classification or regression tasks (which will be discussed in Section 5.2), we run a 10-fold cross validation with the standard Gradient Boosting Classifier or Regressor [Friedman, 2001] in the scikit-learn toolkit [Pedregosa et al., 2011]. |
| Experiment Setup | Yes | We tune Gradient Boosting parameters with a 3-fold cross validation on the training data: learning rate over the range {0.0001 . . . 100} by multiples of 10 and max depth over the range {1 . . . 5}. |