Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
From Persona to Personalization: A Survey on Role-Playing Language Agents
Authors: Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, Yanghua Xiao
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this paper, we conduct a comprehensive survey of this field, illustrating the evolution and recent progress in RPLAs integrating with cutting-edge LLM technologies. We categorize personas into three types: 1) Demographic Persona, which leverages statistical stereotypes; 2) Character Persona, focused on well-established figures; and 3) Individualized Persona, customized through ongoing user interactions for personalized services. We begin by presenting a comprehensive overview of current methodologies for RPLAs, followed by the details for each persona type, covering corresponding data sourcing, agent construction, and evaluation. Afterward, we discuss the fundamental risks, existing limitations, and prospects of RPLAs. |
| Researcher Affiliation | Collaboration | 1Fudan University 2Shanghai University 3Wuhan University 4UC Santa Barbara 5System, Inc. |
| Pseudocode | No | The paper describes various methodologies and frameworks from other research but does not present any structured pseudocode or algorithm blocks developed by the authors of this survey paper. |
| Open Source Code | No | The paper refers to open-source projects like Chat Haruhi (Li et al., 2023a) and Role LLM (Wang et al., 2024a) in Appendix A.1, but it does not provide concrete access to source code for the methodology or findings presented in this survey paper itself. |
| Open Datasets | Yes | Table 3: Overview of existing role-playing datasets with individualized personas. Datasets #Profile #Interactions Domain Lang. Source PERSONA-CHAT (Zhang et al., 2018) 1,155 10,907 EN Crowdsourcing Conv AI (Dinan et al., 2020) 1,155 17,878 EN Crowdsourcing |
| Dataset Splits | No | As a survey paper, the authors do not conduct new experiments requiring dataset splits. The paper discusses datasets used in other research but does not specify train/test/validation splits for any experiments of its own. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running any experiments, as it is a survey of existing work rather than an experimental paper. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers needed to replicate any experiments, as it is a survey of existing work rather than an experimental paper. |
| Experiment Setup | No | The paper does not describe any specific experimental setup details (e.g., hyperparameter values, training configurations, system-level settings) because it is a survey paper and does not present new experimental results. |