Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
AdaPlanner: Adaptive Planning from Feedback with Language Models
Authors: Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai, Chao Zhang
NeurIPS 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments in the ALFWorld and Mini Wo B++ environments demonstrate that Ada Planner outperforms state-of-the-art baselines by 3.73% and 4.11% while utilizing 2x and 600x fewer samples, respectively. |
| Researcher Affiliation | Academia | Haotian Sun1 , Yuchen Zhuang1 , Lingkai Kong1, Bo Dai1, Chao Zhang1 1 Georgia Institute of Technology EMAIL, EMAIL |
| Pseudocode | Yes | Figure 2: An illustrative example from ALFWorld to show the proposed adaptive closed-loop planning through code. The task is to put some clean lettuce on the diningtable... (a) Initial Plan (Extracted) def solution(agent, start_from=3):... |
| Open Source Code | Yes | The implementation of Ada Planner is available at https://github.com/haotiansun14/Ada Planner. |
| Open Datasets | Yes | We test Ada Planner on two text-based decision-making environments: 1) ALFWorld [20] is a text-based virtual household environment... 2) Mini Wo B++ [11] is a simulation environment... |
| Dataset Splits | No | The paper refers to |
| Hardware Specification | No | The paper mentions using specific LLM models (e.g., GPT-3.5, text-davinci-002) but does not provide details on the hardware (CPU, GPU, RAM) used to run the experiments or host these models. |
| Software Dependencies | Yes | RCI and Ada Planner harness GPT-3.5 (gpt-3.5-turbo and text-davinci-003) as backends. |
| Experiment Setup | Yes | The Setup details and prompts for Ada Planner are depicted in Appendix A and D. |