Agent Design Consistency Checking via Planning
Authors: Nitin Yadav, John Thangarajah, Sebastian Sardina
IJCAI 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments in more than 16K random instances prove that the approach is more effective than previous ones proposed: it achieves higher coverage, lower run-time, and importantly, can handle loops in the agent detailed design and unbounded subgoal reasoning. |
| Researcher Affiliation | Academia | Nitin Yadav University of Melbourne, Australia nitin.yadav@unimelb.edu.au John Thangarajah and Sebastian Sardina RMIT University, Australia {john.thangarajah, sebastian.sardina}@rmit.edu.au |
| Pseudocode | No | The paper presents PDDL action definitions which are structured similarly to code, but these are not explicitly labeled as "Pseudocode" or an "Algorithm" block. |
| Open Source Code | No | The paper provides links to third-party tools (fast-downward, MCMAS, Nu SMV) used in the experiments but does not provide concrete access to the authors' own implementation code for their proposed methodology. |
| Open Datasets | No | The paper describes the generation of its own benchmark: "The benchmark was constructed in two steps (i) build agent details; and (ii) for each agent detail generate multiple scenarios... In total the benchmark had 16, 088 instances." However, it does not state that this generated benchmark is publicly available or provide a link/citation for access. |
| Dataset Splits | No | The paper describes the generation of its test cases but does not provide specific details on training, validation, and test splits (e.g., percentages, sample counts, or explicit mention of cross-validation). |
| Hardware Specification | Yes | Experiments were conducted on a machine with 4Ghz corei7 CPU with 32GB RAM. |
| Software Dependencies | No | The paper mentions the use of "fast-downward", "MCMAS", and "Nu SMV" but does not specify their version numbers. |
| Experiment Setup | Yes | The benchmark was constructed in two steps (i) build agent details; and (ii) for each agent detail generate multiple scenarios. The agent details were randomly generated based on three parameters: the height of the design (denoted by h), the maximum number of goals per plan (denoted by g/p), and the maximum number of plans per goal (denoted by p/g)... A time limit of 10 minutes per problem instance was used for all the solvers. As a planner we used fast-downward with ff as the heuristic with lazy search. |