Learning From Unannotated QA Pairs to Analogically Disambiguate and Answer Questions
Authors: Maxwell Crouse, Clifton McFate, Kenneth Forbus
AAAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the efficacy of our technique using the Geoquery corpus, on which it approaches state of the art performance using 10-fold cross validation, shows little decrease in performance with 2folds, and achieves above 50% accuracy with as few as 10 examples. We present our results in comparison to other high performing Geoquery systems in Table 1. Consistent with several prior systems (e.g. Wong and Mooney 2007; Lu et al. 2008) we evaluate using 10-fold cross-validation over the entire set of 880 questions. |
| Researcher Affiliation | Academia | Maxwell Crouse, Clifton Mc Fate, Kenneth Forbus Qualitative Reasoning Group, Northwestern University mvcrouse@u.northwestern.edu; {c-mcfate, forbus}@northwestern.edu |
| Pseudocode | Yes | Figure 6: Query Composition Algorithm |
| Open Source Code | No | The paper does not include an unambiguous statement or a direct link indicating that the authors are releasing the source code for the work described in this paper. |
| Open Datasets | Yes | We demonstrate the efficacy of our technique using the Geoquery corpus |
| Dataset Splits | Yes | Consistent with several prior systems (e.g. Wong and Mooney 2007; Lu et al. 2008) we evaluate using 10-fold cross-validation over the entire set of 880 questions. We also evaluate using 2-fold cross-validation. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions several software components like "Research Cyc", "EA NLU semantic parser", and "Prolog interpreter", but it does not provide specific version numbers for any of these or other ancillary software components. |
| Experiment Setup | No | The paper describes the overall technique and algorithms but does not provide specific experimental setup details such as hyperparameter values, learning rates, batch sizes, or other concrete training configurations necessary for replication. |