Hieros: Hierarchical Imagination on Structured State Space Sequence World Models
Authors: Paul Mattes, Rainer Schlosser, Ralf Herbrich
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that our approach outperforms the state of the art in terms of mean and median normalized human score on the Atari 100k benchmark, and that our proposed world model is able to predict complex dynamics very accurately. We also show that HIEROS displays superior exploration capabilities compared to existing approaches. We conduct a thorough ablation study that combining hierarchical imagination based learning and our S5WM yields superior results. |
| Researcher Affiliation | Academia | Paul Mattes 1 Rainer Schlosser 1 Ralf Herbrich 1 1Digital Engineering Faculty, Hasso Plattner Institute, University of Potsdam, Germany. Correspondence to: Paul Mattes <paul.mattes@student.hpi.de>, Rainer Schlosser <rainer.schlosser@hpi.de>. |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | Yes | We provide the source code in the supplementary material and under the following link: https://github.com/Snagnar/Hieros. |
| Open Datasets | Yes | We evaluate the performance of HIEROS on the Atari100k test suite (Bellemare et al., 2013). |
| Dataset Splits | No | The paper mentions evaluating on the Atari100k test suite and running multiple training runs for ablations, but it does not specify explicit train/validation/test dataset splits (e.g., percentages or sample counts) for the data used within each run, nor does it reference predefined splits with citations for such partitioning. |
| Hardware Specification | Yes | In our experiments we use a machine with an NVIDIA A100 graphics card with 40 GB of VRAM, 8 CPU cores and 32 GB RAM. |
| Software Dependencies | No | We base our implementation on the Pytorch implementation of Dreamer V3 (NM512, 2023) and on the Pytorch implementation of the S5 layer (C2D, 2023). While specific GitHub repositories are cited, explicit version numbers for general software dependencies like PyTorch, Python, or CUDA are not provided. |
| Experiment Setup | Yes | We use the hyperparameters as specified in Appendix E for all experiments. Appendix E lists detailed 'Training parameters' (e.g., learning rate 10^-4, batch size 16, imagination horizon 16) and 'World Model parameters' (e.g., S5 model dimension 256, Number of S5 blocks 8). |