Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Why Can’t You Do That HAL? Explaining Unsolvability of Planning Tasks
Authors: Sarath Sreedharan, Siddharth Srivastava, David Smith, Subbarao Kambhampati
IJCAI 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical analysis and user studies show the validity of our methods as well as their computational efficacy on a number of benchmark planning domains. |
| Researcher Affiliation | Collaboration | 1CIDSE, Arizona State University, Tempe, AZ 85281 USA 2PSresearch EMAIL, EMAIL, EMAIL, EMAIL |
| Pseudocode | No | The paper describes methods and formulations but does not include a clearly labeled 'Pseudocode' or 'Algorithm' block with structured steps. |
| Open Source Code | No | The paper provides a link to supplementary material for study setup ('http://bit.ly/2HQ5sTv') but does not provide a link or explicit statement about the availability of the source code for its described methodology. |
| Open Datasets | Yes | The first five domains and their problem instances consisted of standard IPC domains and problem instances used in previous IPC competitions [International Planning Competition, 2011]. ... The next three domains were selected from the set used for the 2016 unsolvability competition [Unsolvability International Planning Competition, 2016]. |
| Dataset Splits | No | The paper evaluates its methods on standard planning benchmarks but does not specify explicit training, validation, or test dataset splits for these problems. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU or CPU models, or cloud computing specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions using 'the fast-downward implementation of [Keyder, Richter, and Helmert, 2010]' but does not provide a specific version number for this or any other software dependency. |
| Experiment Setup | Yes | All instances were run with a timeout of 100 minutes (all problems were solvable under this time limit) and all landmarks were generated using the fast-downward implementation of [Keyder, Richter, and Helmert, 2010] (where we set the subset sizes to one for the first five domains and to two for the rest). |