Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation
Authors: Yudan Wang, Yue Wang, Yi Zhou, Shaofeng Zou
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical results are also provided in the appendix. In this section, we conduct experiments to numerically verify our AC/NAC with compatible function approximation. |
| Researcher Affiliation | Academia | 1Electrical Engineering, University at Buffalo 2Electrical and Computer Engineering, University of Central Florida 3Electrical and Computer Engineering, University of Utah 4Computer Science & Engineering, University at Buffalo. |
| Pseudocode | Yes | Algorithm 1 (Natural) Actor-Critic with Compatible Function Approximation; Algorithm 2 Compatible k-step TD Algorithm |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described. |
| Open Datasets | Yes | We test our algorithms in the Acrobot environment (Sutton, 1995). |
| Dataset Splits | No | The paper uses the Acrobot environment but does not specify explicit training, validation, or test dataset splits. |
| Hardware Specification | No | The paper does not provide specific hardware details used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers. |
| Experiment Setup | Yes | In our experiment setup, we set k = 128, and design a 2-layer neural network with 16 hidden neurons to represent the policy, which contains 163 parameters. |