Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Proportional Aggregation of Preferences for Sequential Decision Making
Authors: Nikhil Chandak, Shashwat Goel, Dominik Peters
AAAI 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present empirical results for these rules based on synthetic data and U.S. political elections. We also run experiments using the moral machine dataset about ethical dilemmas. |
| Researcher Affiliation | Academia | Nikhil Chandak1, Shashwat Goel1, Dominik Peters2 1IIIT Hyderabad 2CNRS, LAMSADE, Université Paris Dauphine PSL |
| Pseudocode | No | The paper describes the methods (Sequential Phragmén, Method of Equal Shares, Proportional Approval Voting) in prose but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper mentions data availability ('available at https://osf.io/t6p7s/') but does not provide a specific link or explicit statement about the release of its own source code for the methodology described. |
| Open Datasets | Yes | In addition to synthetic data, we evaluated the rules on data from U.S. political general elections which we collected (available at https://osf.io/t6p7s/). [...] to empirically test our work on a dataset that has structured features which allow preference learning, we consider virtual democracy applied to the moral machine (Awad et al. 2018). |
| Dataset Splits | No | The paper mentions training models but does not explicitly provide specific percentages, absolute sample counts, or predefined splits for training, validation, and testing datasets. |
| Hardware Specification | No | The paper does not explicitly describe any specific hardware components such as GPU or CPU models, memory, or cloud instance types used for running experiments. |
| Software Dependencies | No | The paper discusses the use of the Plackett-Luce (PL) model but does not specify version numbers for any software dependencies or libraries used in the implementation. |
| Experiment Setup | Yes | For analysis based on synthetic data, we follow a similar setup to the one used by Lackner (2020) which is based on the popular approach of sampling both voters and alternatives as points in a two-dimensional Euclidean space (Elkind et al. 2017). We use n = 20 voters who are split into a group of 6 and a group of 14 voters. [...] We use T = 20 rounds with 20 alternatives per round. [...] We collected instances from 16 counties in California and Colorado from 2020 and 2022 (for which data was available). [...] As a baseline, we train a combined model on respondents from all countries, using 100 samples from each country for a balanced representation in the training data. [...] We produce 100 decision rounds together with 100 alternatives for each round. |