Enhancing Robustness in Deep Reinforcement Learning: A Lyapunov Exponent Approach
Authors: Rory Young, Nicolas Pugeault
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we investigate the impact that the proposed MLE regularisation has on Dreamer V3. We show that the inclusion of this term reduces the chaotic state dynamics produced by the control policy and that this improved stability increases performance when noise is introduced. For these experiments, we train three instances of Dreamer V3 with MLE regularisation and reuse the SAC, TD3 and Dreamer V3 policies from Sections 3 & 4. |
| Researcher Affiliation | Academia | Rory Young Nicolas Pugeault School of Computing Science University of Glasgow |
| Pseudocode | Yes | Algorithm 1 MLE regularisation |
| Open Source Code | Yes | This paper provides open access to the code, with instructions to faithfully reproduce the main experimental results outlined in Sections 3 & 5 and Appendix A.2 & B. |
| Open Datasets | Yes | To closely match real-world applications, we estimate the stability of tasks from the Deep Mind Control Suite [27] when controlled by deep RL policies. |
| Dataset Splits | No | The paper does not explicitly provide training/test/validation dataset splits. It mentions 'evaluation episodes' but not a distinct validation set or split for hyperparameter tuning or model selection during training. |
| Hardware Specification | Yes | All models are based on the Stable Baselines 3 [18] implementation with the parameters outlined in Appendix A.2 and trained using an Intel Core i7-8700 CPU workstation with an Nvidia RTX 2080 Ti GPU and 32GB of RAM. |
| Software Dependencies | No | The paper mentions using 'Stable Baselines 3' but does not provide a specific version number for this software or any other key software components used in the experiments. |
| Experiment Setup | Yes | All models are based on the Stable Baselines 3 [18] implementation with the parameters outlined in Appendix A.2 and trained using an Intel Core i7-8700 CPU workstation with an Nvidia RTX 2080 Ti GPU and 32GB of RAM. |