Robust Consensus in Ranking Data Analysis: Definitions, Properties and Computational Issues
Authors: Morgane Goibert, Clément Calauzènes, Ekhine Irurozki, Stephan Clémençon
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Beyond the theoretical contributions, the relevance of the approach proposed is supported by an experimental study. [...] In this section, we illustrate the relevance of the statistic outputted by our Downward Merge plug-in on Kemeny s median (called our Downward Merge statistic for short) by running several illustrative experiments for various settings and comparing with the baseline provided by the usual Kemeny s median. |
| Researcher Affiliation | Collaboration | 1Criteo AI Lab, Paris, France 2T el ecom Paris, Paris, France. Correspondence to: Morgane Goibert <morgane.goibert@gmail.com>. |
| Pseudocode | Yes | Algorithm 1 Na ıve Merge [...] Algorithm 2 Downward Merge |
| Open Source Code | Yes | The code is available here. |
| Open Datasets | Yes | To corroborate these findings, we also ran experiments using real-world datasets from the preflib library: two Netflix Prize datasets (resp. with n = 3 and n = 4 items), a Debian dataset (with n = 5 items) and an Apa dataset (with n = 5 items). |
| Dataset Splits | No | The paper mentions using "various distributions p" and "real-world datasets" but does not specify any training, validation, or test dataset splits, percentages, or procedures for partitioning the data. |
| Hardware Specification | No | The paper does not provide any details about the hardware (e.g., GPU/CPU models, memory, or specific computing environments) used to run the experiments. |
| Software Dependencies | No | The paper mentions using the "preflib library" but does not specify any software names with version numbers (e.g., Python version, specific library versions, or frameworks like PyTorch or TensorFlow). |
| Experiment Setup | Yes | When the threshold is set to a sensible value (here θ = 0.05), the Downward Merge algorithm outputs a bucket order as a statistic: thus, the robustness increases very strongly to reach nearly optimal values even for very small values of δ, which illustrates its efficiency. [...] Figure 5 shows the results for different choices of distribution p when the number of items n = 4, and for δ = 1/6 (normalized value of δ that requires at least a switch between two items to break the statistic). |