Forethought and Hindsight in Credit Assignment
Authors: Veronica Chelu, Doina Precup, Hado P. van Hasselt
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical studies work to uncover the distinctions between planning in anticipation and in retrospect. With the aim of understanding the underlying properties of these approaches, we ask the following questions: (i) How are the two planning algorithms distinct? When does it matter? The experiments are ablation studies of the effects of varying the fan-in (nx), the fan-out (ny) and the number of levels l with their corresponding sizes nlz. For this investigation we performed two types of experiments. |
| Researcher Affiliation | Collaboration | Veronica Chelu Mila, Mc Gill University Doina Precup Mila, Mc Gill University, Deep Mind Hado van Hasselt |
| Pseudocode | Yes | Algorithm 1: Backward Planning (on page 3), Algorithm 2: Online Forward-Dyna: Learning, Acting & Forward Planning (on page 5), Algorithm 3: Online Backward-Dyna: Learning, Acting & Backward Planning (on page 5) |
| Open Source Code | No | The paper does not provide a statement or a link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | No | The paper describes experimental environments like 'Markov Reward Processes' and a 'discrete navigation task from (Sutton and Barto, 2018)' but does not provide concrete access information (e.g., link, DOI, specific citation for a dataset) for a publicly available or open dataset used in training. |
| Dataset Splits | No | The paper describes experimental setups and algorithms for online learning but does not specify training, validation, or test dataset splits (e.g., percentages or sample counts) needed for reproduction, as it is not a fixed dataset experiment. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments, such as GPU/CPU models, memory, or cloud resources. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies, libraries, or solvers used in the experiments. |
| Experiment Setup | No | The paper describes the general 'Experimental setup' for different studies (e.g., prediction setting, control setting, Markov Reward Processes characteristics) but does not provide specific numerical hyperparameters (e.g., learning rates, batch sizes, number of epochs) or system-level training settings. |