Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Finding Friend and Foe in Multi-Agent Games
Authors: Jack Serrino, Max Kleiman-Weiner, David C. Parkes, Josh Tenenbaum
NeurIPS 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical game-theoretic methods show that Deep Role outperforms other hand-crafted and learned agents in ο¬ve-player Avalon. Deep Role played with and against human players on the web in hybrid human-agent teams. We ο¬nd that Deep Role outperforms human players as both a cooperator and a competitor. |
| Researcher Affiliation | Collaboration | Jack Serrino MIT EMAIL Max Kleiman-Weiner Harvard, MIT, Diffeo EMAIL David C. Parkes Harvard University EMAIL Joshua B. Tenenbaum MIT, CBMM EMAIL |
| Pseudocode | Yes | See Appendix A and Alg. 3 for details of the network training algorithm, procedure, parameters and compute details. |
| Open Source Code | Yes | Source code and experimental data is available here: https://github.com/Detry322/Deep Role. |
| Open Datasets | No | The paper does not provide concrete access information (link, DOI, repository, or formal citation) for a publicly available or open dataset. It describes generating its own training data through self-play and human interactions but does not make this data publicly accessible. |
| Dataset Splits | No | The paper does not provide specific dataset split information (percentages, sample counts, or citations to predefined splits) needed to reproduce the data partitioning. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | Yes | Our network feeds a one-hot encoded vector of the proposer player i and the belief vector b into two fully-connected hidden layers of 80 Re LU units. These feed into a fully-connected win probability layer with sigmoid activation. |