Can Large Language Model Agents Simulate Human Trust Behavior?
Authors: Chengxing Xie, Canyu Chen, Feiran Jia, Ziyu Ye, Shiyang Lai, Kai Shu, Jindong Gu, Adel Bibi, Ziniu Hu, David Jurgens, James Evans, Philip Torr, Bernard Ghanem, Guohao Li
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we focus on one critical and elemental behavior in human interactions, trust, and investigate whether LLM agents can simulate human trust behavior. We first find that LLM agents generally exhibit trust behavior, referred to as agent trust, under the framework of Trust Games, which are widely recognized in behavioral economics. Then, we discover that GPT-4 agents manifest high behavioral alignment with humans in terms of trust behavior, indicating the feasibility of simulating human trust behavior with LLM agents. |
| Researcher Affiliation | Collaboration | 1KAUST 2Illinois Institute of Technology 3University of Oxford 4Pennsylvania State University 5University of Chicago 6Emory 7California Institute of Technology 8University of Michigan 9Santa Fe Institute 10Google 11CAMEL-AI.org |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Project website: https://agent-trust.camel-ai.org |
| Open Datasets | Yes | Comparing the results of LLM agents with existing human studies in Trust Games, we have our second core finding: GPT-4 agents manifest high behavioral alignment with humans in terms of trust behavior... |
| Dataset Splits | No | The paper does not describe specific training, validation, and test splits for a dataset. Instead, it uses pre-trained Large Language Models as agents and compares their behavior to existing human studies in various Trust Games, where data is observed rather than split for model training purposes. |
| Hardware Specification | Yes | For the open-source LLMs (e.g., Llama-7B), we conduct model inference in a NVIDIA RTX A6000. |
| Software Dependencies | No | The paper mentions using the CAMEL framework and various LLM models (e.g., GPT-4, Llama2), and indicates using OpenAI APIs for closed-source models. However, it does not specify version numbers for general software dependencies, programming languages, or libraries like Python, PyTorch, or TensorFlow. |
| Experiment Setup | Yes | We set the temperature as 1 to increase the diversity of agents decision-making and note that high temperatures are commonly adopted in related literature (Aher et al., 2023; Lorè & Heydari, 2023; Guo, 2023). |