Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Adaptable Agent Populations via a Generative Model of Policies
Authors: Kenneth Derek, Phillip Isola
NeurIPS 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We test our generative model s capabilities in a variety of environments, including an open-ended grid-world and a two-player soccer environment. |
| Researcher Affiliation | Academia | Kenneth Derek MIT CSAIL EMAIL Phillip Isola MIT CSAIL EMAIL |
| Pseudocode | Yes | See Algorithm 1 in the supplement for additional details. [...] Details can be found in Algorithm 2 of the supplement. |
| Open Source Code | Yes | Code, visualizations, and additional experiments can be found at https://kennyderek.github.io/adap/. |
| Open Datasets | No | The paper describes using simulated environments ('Farmworld' and 'Markov Soccer') which generate data dynamically. It does not provide access information for a static, publicly available dataset. |
| Dataset Splits | No | The paper refers to training and testing in ablated environments but does not provide specific details on dataset splits (e.g., percentages or sample counts) for training, validation, or test sets in the traditional sense of static datasets. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models or processor types used for running experiments. |
| Software Dependencies | No | The paper mentions frameworks like PPO and DIAYN but does not list specific software dependencies with version numbers (e.g., Python 3.8, PyTorch 1.9). |
| Experiment Setup | No | The paper states, 'Training and Hyperparameters We train each method for the same number of timesteps (30 million), and generally keep hyperparameters constant across methods. These are described in the supplement.' Since the details are deferred to the supplement, they are not explicitly in the main text. |