Learning Successor Features the Simple Way
Authors: Raymond Chua, Arna Ghosh, Christos Kaplanis, Blake Richards, Doina Precup
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that our approach matches or outperforms existing SF learning techniques in both 2D (Minigrid), 3D (Miniworld) mazes and Mujoco, for both single and continual learning scenarios. |
| Researcher Affiliation | Collaboration | Raymond Chua Arna Ghosh Christos Kaplanis Blake A. Richards Doina Precup ... Dept of Neurology & Neurosurgery, and Montreal Neurological Institute of Mc Gill University. Co-senior Authorship. CIFAR Learning in Machines and Brains. School of Computer Science, Mc Gill University & Mila Google Deepmind |
| Pseudocode | Yes | The full algorithm used for training our network is given in Algorithm 1 in Appendix B. |
| Open Source Code | Yes | 1Code: https://github.com/raymondchua/simple_successor_features |
| Open Datasets | Yes | The 2D Gridworld environments were developed based on 2D Minigrid [Chevalier-Boisvert et al., 2023]. We developed the 3D Four Rooms environments (Figure 9d) using Miniworld [Chevalier-Boisvert et al., 2023]. |
| Dataset Splits | No | The paper provides training parameters like 'NUM TRAINING FRAMES PER TASK' but does not specify explicit training, validation, and test dataset splits in the typical supervised learning sense. |
| Hardware Specification | Yes | All experiments, particularly those in the continual learning setting, were conducted using Nvidia V100 GPUs and completed within a maximum of one day. |
| Software Dependencies | Yes | For our experimental setup, we utilized Python 3 [Van Rossum and Drake, 2009] as the primary programming language. The agent creation and computational components were developed using Jax [Bradbury et al., 2018, Godwin* et al., 2020], while Haiku [Hennigan et al., 2020] was employed for implementing the neural network components. For data visualization, we used Matplotlib [Hunter, 2007] and Seaborn [Waskom, 2021]... We utilized Scikit-learn [Pedregosa et al., 2011]... UMAP) tool [Mc Innes et al., 2018]... Hydra [Yadan, 2019] and Weights & Biases [Biewald, 2020]. |
| Experiment Setup | Yes | The specific parameters defining the 2D Gridworld environments are detailed in Table 1. The specific parameters defining the 3D Miniworld environments are detailed in Table 2... Detailed hyperparameters for learning SFs and the task encoding w for our agent are outlined in Tables 4 and 5. |