Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Forward Modeling for Partial Observation Strategy Games - A StarCraft Defogger
Authors: Gabriel Synnaeve, Zeming Lin, Jonas Gehring, Dan Gant, Vegard Mella, Vasil Khalidov, Nicolas Carion, Nicolas Usunier
NeurIPS 2018 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experiments We hypothesize that our models will be able to make predictions of the global build tree and local unit numbers better than strong existing baselines. We evaluate our models on a human games dataset, and compare them to heuristic baselines that are currently employed by competitive rule-based bots. We also test whether we are able to use the defogger directly in a state-of-the-art rule-based Star Craft bot, and we evaluate the impact of the best models within full games. |
| Researcher Affiliation | Industry | Gabriel Synnaeve Facebook, NYC EMAIL Zeming Lin Facebook, NYC EMAIL Jonas Gehring Facebook, Paris EMAIL Dan Gant Facebook, NYC EMAIL Vegard Mella Facebook, Paris EMAIL Vasil Khalidov Facebook, Paris EMAIL Nicolas Carion Facebook, Paris EMAIL Nicolas Usunier Facebook, Paris EMAIL |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | We release the code necessary for the reproduction of the project at https://github.com/facebookresearch/starcraft_defogger. |
| Open Datasets | Yes | Our models are trained and evaluated on the STARDATA corpus, which consists of 65,000 high quality human games of Star Craft: Brood War [19]. |
| Dataset Splits | Yes | We use the train, valid, and test splits given by the authors, which comprise 59060, 3289 and 3297 games, respectively. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | Our models are implemented in Py Torch [21], and data preprocessing is done with Torch Craft [22]. (While tools are mentioned, specific version numbers for these software dependencies are not provided.) |
| Experiment Setup | No | We explored the following properties: kernel width of convolutions and striding (3,5); model depth; non-linearities (ReLU, GLU); residual connections; skip connections in the encoder LSTM; optimizers (Adam, SGD); learning rates. Please check the appendix for a more detailed description of hyperparameters searched over. |