How to Evaluate Behavioral Models
Authors: Greg d'Eon, Sophie Greenwood, Kevin Leyton-Brown, James R. Wright
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Researchers building behavioral models, such as behavioral game theorists, use experimental data to evaluate predictive models of human behavior. However, there is little agreement about which loss function should be used in evaluations... We also demonstrate many of these axiom violations on real behavioral data in the appendix. |
| Researcher Affiliation | Academia | 1University of British Columbia 2Cornell University 3University of Alberta |
| Pseudocode | No | The paper is theoretical and axiomatic, focusing on definitions and theorems; it does not include pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any information or links regarding the release of source code for the methodology or analysis described. |
| Open Datasets | No | The paper uses hypothetical examples (e.g., Example 5.1 with p = (2/3, 1/3) and y = (0.6, 0.4)) and refers to 'real behavioral data in the appendix' for demonstrations, but does not provide concrete access information (link, DOI, citation) for any public dataset used in its analysis or examples. |
| Dataset Splits | No | The paper does not provide specific details on training, validation, or test dataset splits; its analysis is primarily theoretical with illustrative examples. |
| Hardware Specification | No | The paper does not specify any hardware used for running its analysis or examples. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers required to reproduce its work. |
| Experiment Setup | No | The paper focuses on theoretical analysis and axiomatic development; it does not describe specific experimental setup details such as hyperparameters or training configurations. |