Estimating $α$-Rank from A Few Entries with Low Rank Matrix Completion
Authors: Yali Du, Xue Yan, Xu Chen, Jun Wang, Haifeng Zhang
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results on evaluating the strategies in three synthetic games and twelve real world games demonstrate that strategy evaluation from a few entries can lead to comparable performance to algorithms with full knowledge of the payoff matrix. |
| Researcher Affiliation | Academia | 1 University College London, UK 2 Institute of Automation, Chinese Academy of Sciences 3Beijing Key Laboratory of Big Data Management and Analysis Methods, GSAI, Renmin University of China. |
| Pseudocode | Yes | Algorithm 1 gives the details of Opt Eval-1. Details of Opt Space algo-rithm are in Appendix A. Algorithm 2 gives the details of Opt Eval-2. |
| Open Source Code | Yes | The demo and code for this project are released under https://github.com/yalidu/optEval.git. |
| Open Datasets | Yes | Gaussian games (Rashid et al., 2021). Bernoulli games (Rowland et al., 2019). Soccer meta-game (Liu et al., 2018). Real world games (Czarnecki et al., 2020) |
| Dataset Splits | No | The paper describes the datasets used (Gaussian, Bernoulli, Soccer meta-game, Real world games) but does not provide explicit training, validation, or test split percentages, sample counts, or references to predefined splits for reproducibility. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments (e.g., GPU/CPU models, memory specifications). |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., Python 3.x, PyTorch 1.x). |
| Experiment Setup | Yes | We evaluate all methods on the finiteregime with = 0.001. Four metrics are considered to evaluate both the correctness of the recovered matrix and the task performance. ... δ is the confidence level on the estimation of payoffs and is set to 0.01. ... To obtain empirical payoffs c Mij, 8(i, j) 2 , agent i and j need to compete against each other for a sufficient number of times. ... we run Opt Eval with a chosen rank r = 5 |