Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Indecision Modeling
Authors: Duncan C. McElfresh, Lok Chan, Kenzie Doyle, Walter Sinnott-Armstrong, Vincent Conitzer, Jana Schaich Borg, John P. Dickerson5975-5983
AAAI 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We test these models using data collected from an online survey where participants choose how to (hypothetically) allocate organs to patients waiting for a transplant. Our Contributions. First, we conduct a pilot experiment to illustrate how different interpretations of indecision lead to different outcomes ( ). To test the utility of these models, we conduct a second experiment to collect a much larger dataset of decision responses ( ). We take a machine learning (ML) perspective, and evaluate each model class based on its goodness-of-fit to this dataset. |
| Researcher Affiliation | Academia | Duncan C Mc Elfresh,1 Lok Chan,2 Kenzie Doyle,3 Walter Sinnott-Armstrong,2 Vincent Conitzer,2 Jana Schaich Borg,2 John P Dickerson1 1 University of Maryland, College Park 2 Duke University 3 University of Oregon |
| Pseudocode | No | The paper describes mathematical models and processes but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | All code used for our analysis is available online, 9 and details of our implementation can be found in Appendix ??. 9https://github.com/duncanmcelfresh/indecision-modeling |
| Open Datasets | No | The paper mentions data collected from an online survey for Study 1 and Study 2 and notes that the datasets collected in Study 1 and Study 2 provide some insight, but it does not provide concrete access information (link, DOI, specific repository, or formal citation with author/year) for these datasets to be publicly available. |
| Dataset Splits | No | For each participant we randomly split their question-response pairs into a training and testing set of equal size (20 responses each). No explicit mention of a validation set split is found. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'scikit-learn' used for the MLP, citing Pedregosa et al. (2011), but does not specify its version number or other software dependencies with versioning. |
| Experiment Setup | Yes | We take a maximum likelihood estimation (MLE) approach to fitting each model: i.e., we select agent parameters u and λ which maximize the loglikelihood (LL) of the training responses. Since the LL of these models is not convex, we use random search via a Sobol process (Sobol 1967). The search domain for utility vectors is u [ 1, 1]N, the domain for probability parameters is (0, 1), and the domain for λ depends on the model type (see Appendix ??). |