Constrained Latent Action Policies for Model-Based Offline Reinforcement Learning
Authors: Marvin Alles, Philip Becker-Ehmck, Patrick van der Smagt, Maximilian Karl
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically evaluate C-LAP on the D4RL and V-D4RL benchmark, and show that C-LAP is competitive to state-of-the-art methods, especially outperforming on datasets with visual observations. |
| Researcher Affiliation | Collaboration | 1Machine Learning Research Lab, Volkswagen Group 2Technical University of Munich 3Eötvös Loránd University Budapest |
| Pseudocode | Yes | B Algorithm We provide the general algorithm of C-LAP. Algorithm 1: C-LAP |
| Open Source Code | Yes | 1Code is available at: https://github.com/marvinalles/c-lap |
| Open Datasets | Yes | We empirically evaluate C-LAP on the D4RL and V-D4RL benchmark, and show that C-LAP is competitive to state-of-the-art methods, especially outperforming on datasets with visual observations. ... To focus on the latter, we separately evaluate the performance on low-dimensional feature observations using the D4RL benchmark [26], and on image observations using the V-D4RL benchmark [20]. |
| Dataset Splits | No | The paper describes using D4RL and V-D4RL benchmarks, which typically have predefined train/test splits. However, it does not explicitly state the dataset splits for training, validation, or testing within the paper's text, nor does it refer to predefined splits with specific citations or percentages. It only mentions 'static dataset D = {(o1:T , a1:T , r1:T )N n=1}' for offline reinforcement learning in general. |
| Hardware Specification | Yes | C-LAP experiments with visual observations take around 10 hours on a RTX8000 GPU and experiments with low-dimension feature observations around 11 hours on a A100 GPU. |
| Software Dependencies | Yes | We implement all methods in JAX [33] using Equinox [34]. |
| Experiment Setup | Yes | We provide the hyper-parameters of CLAP in Table 2 and the constraint values used for the D4RL benchmark in Table 3 and for the V-D4RL benchmark in Table 4. |