Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Conditions on Features for Temporal Difference-Like Methods to Converge
Authors: Marcus Hutter, Samuel Yang-Zhao, Sultan Javed Majeed
IJCAI 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | Our main contribution is to prove that natural algorithms, even under the setting where the value function can be represented exactly by the features, are inherently prone to nonuniqueness and will converge to the wrong solution for most feature choices. Our main result is as follows: Theorem 5.1. Natural algorithms converge if and only if all non-zero linear combination of the features achieve their extreme values on a sub-region of the state space that has nonzero measure under the stationary distribution. |
| Researcher Affiliation | Academia | Marcus Hutter1 , Samuel Yang-Zhao1 and Sultan Javed Majeed1 1Australian National University EMAIL |
| Pseudocode | No | The paper contains mathematical formulations, definitions, and theorems but no pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement about the release of source code or links to a code repository for the described methodology. |
| Open Datasets | No | The paper is theoretical and does not use or reference any specific datasets for training experiments. |
| Dataset Splits | No | The paper is theoretical and does not report on experimental validation, thus no dataset splits for validation are provided. |
| Hardware Specification | No | The paper does not provide any specific details about hardware used for experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies or version numbers. |
| Experiment Setup | No | The paper is theoretical and does not describe any experimental setup details such as hyperparameters or training configurations. |