CompeteAI: Understanding the Competition Dynamics of Large Language Model-based Agents
Authors: Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, Xing Xie
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Simulation experiments reveal several interesting findings at the micro and macro levels, which align well with existing market and sociological theories. |
| Researcher Affiliation | Collaboration | 1University of Science and Technology of China 2Microsoft Research 3William & Mary 4Georgia Institute of Technology 5Carnegie Mellon University. |
| Pseudocode | No | The paper provides flowcharts and diagrams (e.g., Figure 7 and 8) to illustrate processes, but it does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at: https: //github.com/microsoft/competeai. |
| Open Datasets | No | The paper describes simulating a virtual town with 2 restaurants and 50 customer agents, where customer characteristics are defined (Appendix C.2). This is a custom-built simulation environment, and the 'data' is generated through the interaction of LLM agents, not loaded from a publicly available dataset in the conventional sense for training or evaluation. |
| Dataset Splits | No | The paper describes a simulation environment and agent interactions using GPT-4. It does not provide traditional training/validation/test dataset splits as it generates data through simulation rather than processing a static dataset. |
| Hardware Specification | No | The paper mentions using GPT-4 (0613) and API fees for running simulations, but it does not specify any particular hardware components such as GPU or CPU models, or memory used for the experiments. |
| Software Dependencies | Yes | In this paper, both restaurants and customers are powered by LLM-based agents, which are GPT-4 (0613) (Open AI, 2023). |
| Experiment Setup | Yes | Based on the framework, we implement the environment as a small town with two types of entities: 2 restaurants and 50 customers. ... We set the simulation runs for 15 days... To gauge the quality of the dishes, we formulate several key assumptions... we introduce an empirical mechanism to evaluate the score s for each dish: s = 0.5 c p + 0.5 f 5000, where c is the cost, p is the price, and f is the salary for the chef. |