Using Narrative Function to Extract Qualitative Information from Natural Language Texts
Authors: Clifton McFate, Kenneth Forbus, Thomas Hinrichs
AAAI 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We further evaluated system performance on the first nine simplified paragraphs from chapter two of a science book intended for general readers (Buckley, 1979). ... Of the 144 tagged frames in the corpus, our system correctly constructed 65. There were 23 extraneous, partial, or incorrect frames generated as well. This gives us a precision of .74 and a recall of .45. The F1 harmonic mean was .56. ... Figures 3 and 4 show the difference in the two conditions, averaged over 10 games. The improvement in population growth (Figure 3) is due to the effect of irrigation, while the improvement in science output (Figure 4) is due to the other improvements. The improvement in science output is statistically significant (p < 0.041)... |
| Researcher Affiliation | Academia | Clifton Mc Fate, Kenneth D. Forbus, Thomas R. Hinrichs Qualitative Reasoning Group, Northwestern University 2133 Sheridan Road, Evanston, IL, 60208, USA mcfateclifton79@gmail.com, {forbus, t-hinrichs}@northwestern.edu |
| Pseudocode | Yes | Table 2: Example of rules involved in computing narrative function. (<== (introduces QPQuantity Frame (Presentation Event Fn ?sid ?nevent) ?qframeid) (quantity Type Of Entity Found ?sid ?qtype ?quantity ?entity ?etype) (builds QPFrame ?sid ?qtype ?quantity ?entity ?etype ?qframeid ?nevent)) |
| Open Source Code | No | The paper does not include an unambiguous statement from the authors about releasing their source code, nor does it provide a direct link to a code repository for the methodology described. It mentions Freeciv is open-source but that refers to the game they used, not their own implementation code. |
| Open Datasets | Yes | We further evaluated system performance on the first nine simplified paragraphs from chapter two of a science book intended for general readers (Buckley, 1979). The sentences were taken from the same corpus previously used by Barbella & Forbus (2011) and follow their simplification paradigm. |
| Dataset Splits | No | The paper does not provide specific dataset split information such as exact percentages or sample counts for training, validation, or test sets. It mentions the total number of tagged frames in the corpus but not how they were partitioned for evaluation. |
| Hardware Specification | No | The paper does not provide specific hardware details such as exact GPU/CPU models, processor types, or memory amounts used for running its experiments. |
| Software Dependencies | No | The paper mentions components like "Allen, J. F. (1994) syntactic parser" and "COMLEX (Grishem et al. 1993)" and "Research Cyc" but does not provide specific version numbers for these or other software dependencies required to replicate the experiments. |
| Experiment Setup | No | The paper describes the setup for the strategy game experiment (e.g., "played for 100 turns on each map under two different conditions", "averaged over 10 games") but does not include specific experimental setup details such as hyperparameters (learning rate, batch size, epochs), optimizer settings, or other concrete configuration steps for their NLU system. |