Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Distributionally Robust $Q$-Learning
Authors: Zijian Liu, Qinxun Bai, Jose Blanchet, Perry Dong, Wei Xu, Zhengqing Zhou, Zhengyuan Zhou
ICML 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Simulation results further demonstrate its strong empirical robustness. |
| Researcher Affiliation | Collaboration | 1New York University, Stern School of Business 2Horizon Robotics Inc., CA, USA 3Department of Management Science and Engineering, Stanford University, CA, USA 4Department of Electrical Engineering and Computer Sciences, UC Berkeley, CA, USA 5Arena Technologies. |
| Pseudocode | Yes | Algorithm 1 Distributionally Robust Q-Learning |
| Open Source Code | No | The paper does not provide any link or explicit statement about the availability of open-source code for the described methodology. |
| Open Datasets | No | The paper describes a custom supply chain model and generates samples from a simulator, indicating a simulated environment rather than a publicly available dataset. |
| Dataset Splits | No | The paper does not specify training, validation, and test dataset splits with percentages or counts. It defines a simulated environment and parameters for the learning algorithm. |
| Hardware Specification | No | The paper mentions 'limit computation resources' but does not provide any specific details about the hardware used (e.g., CPU, GPU models, or memory). |
| Software Dependencies | No | The paper does not list any specific software dependencies with version numbers. |
| Experiment Setup | Yes | In our experiments, due to the limit computation resources, we fix n = 10. Besides, we take h = 1, p = 2, k = 3 and set the discount factor γ = 0.9 with starting from s1 = 0. ... In the simulation, we set δ = 1 as the perturbation parameter. At the k-th step of Algorithm 1, we set the learning rate αk be 1 1+(1 γ)(k 1) to satisfy the Robbins Monro Condition. ... For the parameter ε used in our estimator, we consider ε {0.49, 0.499.0.5, 0.6}. |