Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Analogical Chaining with Natural Language Instruction for Commonsense Reasoning
Authors: Joseph Blass, Kenneth Forbus
AAAI 2017 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The utility of this technique is demonstrated by performance of an implemented system on problems from the Choice of Plausible Alternatives test of commonsense causal reasoning. |
| Researcher Affiliation | Academia | Qualitative Reasoning Group Northwestern University EMAIL EMAIL |
| Pseudocode | No | The paper includes a workflow diagram (Figure 1) but no explicit pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any links to source code or explicitly state that source code for their work is available. |
| Open Datasets | Yes | The current system is specialized to answer choice multiple choice questions like those from the Choice of Plausible Alternatives (COPA) (Roemmele et al. ) test of commonsense reasoning |
| Dataset Splits | No | The paper mentions using the 'COPA training set' but does not specify any details about how data was split for training, validation, or testing, such as percentages or sample counts. |
| Hardware Specification | No | The paper does not specify any hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions using the 'EA NLU system' and 'CycL' but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | No | The paper does not provide specific details about the experimental setup, such as hyperparameter values, optimization settings, or training configurations. |