Rank Aggregation Using Scoring Rules
Authors: Niclas Boehmer, Robert Bredereck, Dominik Peters
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To understand how and whether the three families of methods practically differ from each other, and how they relate to Kemeny s method, we perform extensive simulations based on synthetic data (sampled using the Mallows and Euclidean models). |
| Researcher Affiliation | Academia | 1 Algorithmics and Computational Complexity, Technische Universit at Berlin 2 Institut f ur Informatik, TU Clausthal 3 CNRS, LAMSADE, Universit e Paris Dauphine PSL |
| Pseudocode | No | The paper provides definitions and describes algorithms textually (e.g., 'Definition 3.2 (Sequential-s-Winner; Seq.-s-Winner). Let s be a scoring system. The social preference function Seq.-s-Winner is defined recursively as follows'), but it does not include formal pseudocode blocks or labeled algorithm figures. |
| Open Source Code | Yes | The code of our experiments is available at github.com/n-boehmer/Rank-Aggregation. |
| Open Datasets | No | The paper states, 'We conduct simulations on profiles generated using the Mallows model (Mallows 1957) (as observed by Boehmer et al. (2021) real-world profiles often seem to be close to some Mallows profile).' While the Mallows model is known, the authors generate their own synthetic data and do not provide access to the specific generated datasets used in their experiments via a link or citation to a public repository. |
| Dataset Splits | No | The paper states 'we sampled 10 000 profiles for each norm-φ {0, 0.1, . . . , 0.9, 1}' but does not specify any training, validation, or test dataset splits. The entire set of sampled profiles appears to be used for analysis rather than being partitioned into distinct splits. |
| Hardware Specification | No | The paper describes conducting 'extensive simulations' but does not provide any specific details about the hardware (e.g., CPU, GPU models, memory, or cloud resources) used to run these experiments. |
| Software Dependencies | No | The paper does not list specific software components with their version numbers (e.g., programming languages, libraries, or solvers) that would be needed to reproduce the experiments. |
| Experiment Setup | Yes | To deal with ties in the computation of our rules, each time we sample a ranking profile over candidates C, we also sample a ranking tie L(C) uniformly at random and break ties according to tie for all rules. We conduct simulations on profiles generated using the Mallows model (Mallows 1957). We sampled 10 000 profiles for each norm-φ {0, 0.1, . . . , 0.9, 1}. |