Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Analogical Chaining with Natural Language Instruction for Commonsense Reasoning
Authors: Joseph Blass, Kenneth Forbus
AAAI 2017 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The utility of this technique is demonstrated by performance of an implemented system on problems from the Choice of Plausible Alternatives test of commonsense causal reasoning. |
| Researcher Affiliation | Academia | Qualitative Reasoning Group Northwestern University EMAIL EMAIL |
| Pseudocode | No | The paper includes a workflow diagram (Figure 1) but no explicit pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any links to source code or explicitly state that source code for their work is available. |
| Open Datasets | Yes | The current system is specialized to answer choice multiple choice questions like those from the Choice of Plausible Alternatives (COPA) (Roemmele et al. ) test of commonsense reasoning |
| Dataset Splits | No | The paper mentions using the 'COPA training set' but does not specify any details about how data was split for training, validation, or testing, such as percentages or sample counts. |
| Hardware Specification | No | The paper does not specify any hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions using the 'EA NLU system' and 'CycL' but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | No | The paper does not provide specific details about the experimental setup, such as hyperparameter values, optimization settings, or training configurations. |