Regularized Anderson Acceleration for Off-Policy Deep Reinforcement Learning
Authors: Wenjie Shi, Shiji Song, Hui Wu, Ya-Chu Hsu, Cheng Wu, Gao Huang
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The effectiveness of our method is evaluated on a variety of benchmark tasks, including Atari 2600 and Mu Jo Co. Experimental results show that our approach substantially improves both the learning speed and final performance of state-of-the-art deep RL algorithms. |
| Researcher Affiliation | Academia | Wenjie Shi, Shiji Song, Hui Wu, Ya-Chu Hsu, Cheng Wu, Gao Huang Department of Automation, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology (BNRist) {shiwj16, wuhui14, xuyz17}@mails.tsinghua.edu.cn {shijis, wuc, gaohuang}@tsinghua.edu.cn |
| Pseudocode | Yes | Algorithm 1: RAA-Dueling-DQN Algorithm |
| Open Source Code | Yes | The code and models are available at: https://github.com/shiwj16/raa-drl. |
| Open Datasets | Yes | Atari 2600. For discrete control tasks, we perform experiments in the Arcade Learning Environment. Mu Jo Co. For continuous control tasks, we conduct experiments in environments built on the Mu Jo Co physics engine. |
| Dataset Splits | No | The paper does not explicitly provide specific training, validation, and test dataset splits in terms of percentages or sample counts for static datasets. It describes evaluation rollouts: 'evaluated every 10000 environment steps, where each evaluation reports the average return over ten different rollouts.' |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments (e.g., CPU, GPU models, memory specifications). |
| Software Dependencies | No | The paper mentions software components like Dueling-DQN and TD3 but does not list specific version numbers for any software dependencies, such as programming languages, libraries, or frameworks. It states: 'All default hyperparameters used in these experiments are listed in Appendix C of the supplementary material.', but this does not confirm software versions. |
| Experiment Setup | Yes | All default hyperparameters used in these experiments are listed in Appendix C of the supplementary material. |