Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Beyond Pairwise Comparisons in Social Choice: A Setwise Kemeny Aggregation Problem
Authors: Hugo Gilbert, Tom Portoleau, Olivier Spanjaard1982-1989
AAAI 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our numerical tests have three objectives: we evaluate the computational performance of the dynamic programming approach of Section 3, we evaluate the impact of parameter k on the set of consensus rankings, and we assess the efficiency of the preprocessing technique of Section 4. |
| Researcher Affiliation | Academia | Hugo Gilbert Gran Sasso Science Institute 67100 L Aquila, Italy EMAIL Tom Portoleau LAAS-CNRS, IRIT-CNRS Universit e de Toulouse 31400 Toulouse, France EMAIL Olivier Spanjaard Sorbonne Universit e CNRS, LIP6, 75005 Paris, France EMAIL |
| Pseudocode | No | The paper describes algorithms and mathematical formulations but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any concrete access information for open-source code for the methodology described. |
| Open Datasets | No | The paper states that preference profiles are 'generated according to the Mallows model' using a 'Python package Pref Lib-Tools'. This indicates data generation rather than the use of a pre-existing publicly available dataset with a specific link or citation. |
| Dataset Splits | No | The paper describes data generation and sets 'the number n of voters to 50' and varies 'm, k, φ'. However, it does not specify any explicit training, validation, or test dataset splits, or refer to standard benchmark splits. |
| Hardware Specification | Yes | All times are CPU seconds on an Intel Core I7-8700 3.20 GHz processor with 16GB of RAM. |
| Software Dependencies | No | The paper mentions 'Implementation in C++' and 'using the Python package Pref Lib-Tools (Mattei and Walsh 2013)'. While Python and a package are named, specific version numbers for the package or any other libraries are not provided, which is necessary for a reproducible description. |
| Experiment Setup | Yes | The preference profiles are generated according to the Mallows model (Mallows 1957), using the Python package Pref Lib-Tools (Mattei and Walsh 2013). This model takes two parameters as input: a reference ranking σ (the mode of the distribution) and a dispersion parameter φ (0, 1). ... In all tests, the number n of voters is set to 50 and the ranking σ is set arbitrarily as the k-wise Kemeny rule is neutral. For each triple (m, k, φ) considered, the results are averaged over 50 preference profiles. |