Constraint Detection in Natural Language Problem Descriptions
Authors: Zeynep Kiziltan, Marco Lippi, Paolo Torroni
IJCAI 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate the method, we develop an original annotated corpus which gathers 110 problem descriptions from several resources. Our results show significant accuracy with respect to metrics used in cognate tasks. We performed experiments on our dataset following the leave-one-problem-out (LOO) procedure. |
| Researcher Affiliation | Academia | Zeynep Kiziltan and Marco Lippi and Paolo Torroni Department of Computer Science and Engineering DISI University of Bologna, Italy {zeynep.kiziltan, marco.lippi3, p.torroni}@unibo.it |
| Pseudocode | No | The paper does not contain explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our system together with all the reported predictions are available at: http://nlp4cp.disi.unibo.it |
| Open Datasets | Yes | Being the first ones to tackle constraint detection, we had to construct a dataset, that is, a corpus of NL problem descriptions where the parts of text containing problem constraints are annotated. ... The final dataset6 contains 1,075 sentences, for a total of 25,317 words... http://nlp4cp.disi.unibo.it |
| Dataset Splits | Yes | We performed experiments on our dataset following the leave-one-problem-out (LOO) procedure. This is a standard ML methodology, where each problem in turn is selected as test set while the remaining ones form the training set. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory, or cloud instance types) used for running experiments. |
| Software Dependencies | No | The paper mentions 'Stanford Core NLP library' and 'SVM-HMM', but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | Table 1 reports the results obtained on our dataset by different classifiers, as a function of the diameter D used to build contextual features for each word. ... for each word wj we keep the original (unchanged) term, and we also extract the part-of-speech and the stemmed word, both obtained with the Stanford Core NLP library7. ... Finally, we also add the following bag-of-trigrams both for words and for part-of-speech tags: [wj 2wj 1wj], [wj 1wjwj+1], [wjwj+1wj+2]. |