Fast and Accurate Inference of Plackett–Luce Models
Authors: Lucas Maystre, Matthias Grossglauser
NeurIPS 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4 Experimental evaluation In this section, we compare LSR and I-LSR to other inference algorithms in terms of (a) statistical efficiency, and (b) empirical performance. In order to understand the efficiency of the estimators, we generate synthetic data from a known ground truth. Then, we look at five real-world datasets and investigate the practical performance of the algorithms in terms of accuracy, running time and convergence rate. |
| Researcher Affiliation | Academia | Lucas Maystre EPFL lucas.maystre@epfl.ch Matthias Grossglauser EPFL matthias.grossglauser@epfl.ch |
| Pseudocode | Yes | Algorithm 1 Luce Spectral Ranking Require: observations D 1: λ 0n n 2: for (i, A) D do 3: for j A \ {i} do 4: λji λji + n/|A| 5: end for 6: end for 7: π stat. dist. of Markov chain λ 8: return π |
| Open Source Code | No | As an example, our implementation of LSR fits in ten lines of Python code. This sentence indicates the existence of an implementation but does not provide concrete access, such as a repository link, a statement of public release, or mention of code in supplementary materials. |
| Open Datasets | Yes | The NASCAR [18] and sushi [24] datasets contain multiway partial rankings. The You Tube, GIFGIF and chess datasets2 contain pairwise comparisons. Among those, the chess dataset is particular in that it features 45% of ties; in this case we use the extension of the Bradley Terry model proposed by Rao and Kupper [23]. ... 2 See https://archive.ics.uci.edu/ml/machine-learning-databases/00223/, http://www.gif.gf/ and https://www.kaggle.com/c/chess. |
| Dataset Splits | No | The paper does not explicitly provide specific details on training, validation, and test splits with percentages or sample counts. It describes generating synthetic data and using real-world datasets but lacks explicit splitting information for reproducibility. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'our implementation of LSR fits in ten lines of Python code' but does not provide specific ancillary software details, such as library names with version numbers, needed to replicate the experiment. |
| Experiment Setup | Yes | To assess the statistical efficiency of LSR and other algorithms, we follow the experimental procedure of Hajek et al. [11]. We consider n = 1024 items, and draw θ uniformly at random in [ 2, 2]n. We generate d = 64 full rankings over the n items from a Plackett-Luce model parameterized with π [eθi]. For a given k {21, . . . , 210}, we break down each of the full rankings as follows. First, we partition the items into n/k subsets of size k uniformly at random. Then, we store the k-way rankings induced by the full ranking on each of those subsets. As a result, we obtain m = dn/k statistically independent k-way partial rankings. ... Each algorithm is initialized with π(0) = [1/n, . . . , 1/n] , and convergence is declared when ERMS < 0.01. |