Independent Policy Gradient Methods for Competitive Reinforcement Learning
Authors: Constantinos Daskalakis, Dylan J. Foster, Noah Golowich
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Figure 1: Figures (a) and (b) display plots for one ratio game, and Figures (c) and (d) display plots for another; the games matrices are speciļ¬ed in Appendix D.1. Figures (a) and (c) plot the quantity sign( F(z),z z ) for z 2 2, parameterized as z = (x,1 x,y,1 y); yellow denotes negative and purple denotes positive. The red dot denotes the equilibrium z . Figures (b) and (d) plot convergence of extragradient with learning rate 0.01, initialized at z0 = (1,0,1,0); note that z0 is inside the region in which the MVI does not hold for each problem. The blue line plots the primal-dual gap maxy V (x(i),y ) minx V (x ,y(i)) and the orange line plots the primal gap maxy V (x(i),y ) V (x ,y ). |
| Researcher Affiliation | Academia | Constantinos Daskalakis costis@csail.mit.edu Dylan J. Foster dylanf@mit.edu Noah Golowich nzg@mit.edu Massachusetts Institute of Technology |
| Pseudocode | Yes | Algorithm 1: Two-timescale Stochastic Gradient Descent-A scent (SGD-A) for Two-player Zero-sum Games |
| Open Source Code | No | The paper does not contain any explicit statement about releasing source code for the methodology or provide a link to a code repository. |
| Open Datasets | No | The paper defines and analyzes specific game setups ('ratio game', 'stochastic game') directly within its text (e.g., equation (11) and Appendix D.1). It does not use or provide concrete access information for any external, publicly available datasets. |
| Dataset Splits | No | The paper does not provide specific details about training, validation, or test dataset splits. The experiments are conducted on theoretically defined game structures. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers. |
| Experiment Setup | Yes | Figures (b) and (d) plot convergence of extragradient with learning rate 0.01, initialized at z0 = (1,0,1,0); |