Tackling Sparse Rewards in Real-Time Games with Statistical Forward Planning Methods
Authors: Raluca D. Gaina, Simon M. Lucas, Diego Pérez-Liébana1691-1698
AAAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Results show that longer rollouts and individual lengths, either fixed or responsive to changes in fitness landscape features, lead to a boost of performance in the games during testing without being detrimental to non-sparse reward scenarios. |
| Researcher Affiliation | Academia | Queen Mary University of London, UK {r.d.gaina, simon.lucas, diego.perez}@qmul.ac.uk |
| Pseudocode | Yes | Algorithm 1 Adjusting rollout length dynamically |
| Open Source Code | No | Full result files can be found in a Git Hub repository at: github.com/rdgain/Experiment Data/tree/Sparse Rewards. This link is for 'Experiment Data' (result files), not explicitly for the source code of the methodology itself. |
| Open Datasets | Yes | The General Video Game AI Framework and Competition (GVGAI) (Perez-Liebana et al. 2015; Gaina, Perez-Liebana, and Lucas 2016) offers various challenges within the field of General Video Game Playing (Levine et al. 2013). ... A subset of 20 different games is used in this paper, as analysed in (Gaina et al. 2017b). |
| Dataset Splits | No | The paper describes running '100 runs per game, 20 in each of the 5 levels' for evaluation, but it does not specify explicit training, validation, and test dataset splits in terms of percentages, sample counts, or predefined partitions for model training. |
| Hardware Specification | No | The paper states that 'Forward Model calls were used instead of the typical time budget in GVGAI for two reasons. First, it would ensure consistency in results irrespective of the machine used to run the experiments.' No specific hardware models (CPU, GPU, etc.) or detailed specifications are mentioned. |
| Software Dependencies | No | The paper mentions the use of the General Video Game AI Framework (GVGAI) and algorithms like MCTS and RHEA, but it does not specify any software dependencies (e.g., Python, PyTorch, TensorFlow) with version numbers. |
| Experiment Setup | Yes | Our experiments feature the sample MCTS as provided in the GVGAI Framework. Moreover, we applied the same configuration of parameters to both RHEA and MCTS: a population size of 10 for RHEA, rollout length L of 14, budget of 1000 Forward Model (FM) calls... The length L is then increased by the depth modifier MD = 5 if f Ld falls below the lower limit (SD = 0.05), or is decreased by MD if f Ld is above the upper limit (SD+ = 0.4). |