Generating Diverse Cooperative Agents by Learning Incompatible Policies
Authors: Rujikorn Charakorn, Poramate Manoonpong, Nat Dilokthanakul
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, the proposed method consistently discovers more solutions than baseline methods across various multi-goal cooperative environments. Finally, in multi-recipe Overcooked, we show that our method produces populations of behaviorally diverse agents, which enables generalist agents trained with such a population to be more robust. |
| Researcher Affiliation | Academia | 1VISTEC, Rayong, Thailand 2SDU, Odense, Denmark 3KMITL, Bangkok, Thailand |
| Pseudocode | Yes | More details, including the pseudocode and the extension to more than two players, can be found in App. D. |
| Open Source Code | Yes | The source code is available at https://github.com/51616/marl-lipo. |
| Open Datasets | Yes | One-Step Cooperative Matrix Game (CMG): A game of CMG is defined by a tuple (M, {km}, {rm})... Point Mass Rendezvous (PMR): The environment is based on the Multi-Agent Particle Environment (Lowe et al., 2017; Terry et al., 2020)... Overcooked, a collaborative cooking game, has been used to study the cooperative ability of learned agents in prior works (Carroll et al., 2019; Charakorn et al., 2020; Strouse et al., 2021; Mc Kee et al., 2022). |
| Dataset Splits | No | We do not use any validation method. Instead, we present the results using the best parameters in the main paper. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory, or cloud instance types) used for running the experiments. |
| Software Dependencies | No | The paper mentions using 'MAPPO (Yu et al., 2021)' and 'Pettingzoo' but does not provide specific version numbers for these software components or any other libraries. |
| Experiment Setup | Yes | Table 2: Hyperparameters used by the MAPPO algorithm. Learning rate 0.003 (CMG and PMR) 0.005 (Overcooked), Batch size 100 (CMG) 2,500 (PMR) 10,000 (Overcooked), Epochs 10 (CMG and PMR) 15 (Overcooked), Number of mini-batches 2 (CMG and PMR) 5 (Overcooked), Entropy coefficient 0.0 (CMG) 0.03 (PMR and Overcooked), Discount factor (γ) 0.99, GAE lambda 0.95, Value loss coefficient 0.5, PPO clipping parameter 0.3, Gradient clipping 0.5, Adam epsilon 1e-5. |