Weighted importance sampling for off-policy learning with linear function approximation
Authors: A. Rupam Mahmood, Hado P van Hasselt, Richard S. Sutton
NeurIPS 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We compared the performance of the proposed WIS-LSTD(λ) method with the conventional offpolicy LSTD(λ) by Yu (2010) on two random-walk tasks for off-policy policy evaluation. |
| Researcher Affiliation | Academia | Reinforcement Learning and Artificial Intelligence Laboratory University of Alberta Edmonton, Alberta, Canada T6G 1S2 {ashique,vanhasse,sutton}@cs.ualberta.ca |
| Pseudocode | No | The paper describes the algorithm steps using mathematical equations and textual explanations, but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement about open-source code availability or a link to a code repository for the described methodology. |
| Open Datasets | No | The paper describes the setup for two random-walk tasks but does not refer to a publicly available dataset with concrete access information (link, DOI, formal citation with authors/year). |
| Dataset Splits | No | The paper describes experimental runs and error measurement but does not provide specific train/validation/test dataset splits, percentages, or sample counts, nor does it cite predefined splits for data partitioning. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments (e.g., GPU/CPU models, processor types, memory amounts, or cloud specifications). |
| Software Dependencies | No | The paper does not provide specific software dependency details, such as library names with version numbers, required to replicate the experiment. |
| Experiment Setup | Yes | We tested both algorithms for different values of constant λ, from 0 to 0.9 in steps of 0.1 and from 0.9 to 1.0 in steps of 0.025. The matrix to be inverted in both methods was initialized to I, where the regularization parameter was varied by powers of 10 with powers chosen from -3 to +3 in steps of 0.2. |