Decentralized Single-Timescale Actor-Critic on Zero-Sum Two-Player Stochastic Games
Authors: Hongyi Guo, Zuyue Fu, Zhuoran Yang, Zhaoran Wang
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study the global convergence and global optimality of the actor-critic algorithm applied for the zero-sum two-player stochastic games in a decentralized manner. We prove that the sequence of joint policy generated by our decentralized linear algorithm converges to the minimax equilibrium at a sublinear rate O(K), where K is the number of iterations. To the best of our knowledge, we establish the global optimality and convergence of our decentralized actor-critic algorithm on zero-sum two-player stochastic games with linear function approximations for the first time. |
| Researcher Affiliation | Academia | 1Northwestern University 2Princeton University. |
| Pseudocode | Yes | Algorithm 1 Decentralized Actor Critic on Zero-Sum Two-Player Stochastic Games |
| Open Source Code | No | The paper does not provide any statements about open-sourcing code or links to repositories. |
| Open Datasets | No | The paper focuses on theoretical analysis and does not mention using specific datasets for training or their availability. |
| Dataset Splits | No | The paper focuses on theoretical analysis and does not describe any training, validation, or test dataset splits. |
| Hardware Specification | No | The paper focuses on theoretical analysis and does not mention any specific hardware used for experiments. |
| Software Dependencies | No | The paper focuses on theoretical analysis and does not mention any specific software dependencies with version numbers. |
| Experiment Setup | No | The paper focuses on theoretical analysis and does not describe any experimental setup details such as hyperparameters or training configurations. |