Hierarchical State Abstraction based on Structural Information Principles
Authors: Xianghua Zeng, Hao Peng, Angsheng Li, Chunyang Liu, Lifang He, Philip S. Yu
IJCAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical evaluations on a visual gridworld domain and six continuous control benchmarks demonstrate that, compared with five SOTA state abstraction approaches, SISA significantly improves mean episode reward and sample efficiency up to 18.98 and 44.44%, respectively. |
| Researcher Affiliation | Collaboration | Xianghua Zeng1, Hao Peng1, Angsheng Li1,2, Chunyang Liu3, Lifang He4, Philip S. Yu5 1 State Key Laboratory of Software Development Environment, Beihang University 2 Zhongguancun Laboratory 3 Didi Chuxing 4 Department of Computer Science & Engineering, Lehigh University 5 Department of Computer Science, University of Illinois at Chicago {zengxianghua, penghao, angsheng}@buaa.edu.cn, liangsheng@gmail.zgclab.edu.cn, liuchunyang@didiglobal.com, lih319@lehigh.edu, psyu@uic.edu. |
| Pseudocode | Yes | Algorithm 1: The Iterative Optimization Algorithm |
| Open Source Code | Yes | All source codes and experimental results are available at Github1. 1https://github.com/Ring BDStack/SISA |
| Open Datasets | Yes | First, we evaluate SISA for offline state abstraction in a visual gridworld domain, where each discrete position is mapped to a noisy image, like experiments in Markov abstraction [Allen et al., 2021]." and "Next, we benchmark our framework in an online setting with a challenging and diverse set of image-based, continuous control tasks from the Deep Mind Control suite (DMControl) [Tunyasuvunakool et al., 2020]. |
| Dataset Splits | No | The paper states it uses the visual gridworld domain and DMControl suite but does not explicitly provide details on how the data was split into training, validation, and test sets (e.g., percentages, counts, or specific predefined splits). |
| Hardware Specification | Yes | All experiments are conducted on a 3.00GHz Intel Core i9 CPU and an NVIDIA RTX A6000 GPU. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., 'Python 3.8, PyTorch 1.9'). |
| Experiment Setup | Yes | By default, we set the maximal encoding tree height in SISA as 3, K = 3. |