Modeling Human Trust and Reliance in AI-Assisted Decision Making: A Markovian Approach
Authors: Zhuoyan Li, Zhuoran Lu, Ming Yin
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Evaluations on real human behavior data collected from human-subject experiments show that the proposed model outperforms various baselines in accurately predicting humans reliance behavior in AI-assisted decision making. |
| Researcher Affiliation | Academia | Purdue University, USA li4178@purdue.edu, lu800@purdue.edu, mingyin@purdue.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper states "(see supplemental materials for information of the data repository)" but does not mention the release of source code for the methodology described. |
| Open Datasets | Yes | The profiles that we showed to subjects were taken from the UCI Income dataset. |
| Dataset Splits | Yes | To do so, we conduct a 5-fold cross-validation We randomly divide all subjects into five groups and split the behavior dataset into five folds accordingly. Within each cross validation iteration, we use four folds to train the models with grid search being taken to find the best (hyper-)parameters for baseline models, while the learned model s performance is evaluated on the remaining fold. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU or CPU models used for running the experiments. |
| Software Dependencies | No | The paper mentions logistic regression models, XGBoost, and LSTM but does not provide specific version numbers for these software components or libraries. |
| Experiment Setup | Yes | We also experiment with a range of values for the number of hidden trust states (i.e., K = 2 6) and find the model with K = 3 achieves the maximum Bayesian information criterion (BIC) score (Schwarz 1978), thus we set K = 3 throughout our evaluation. |