Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Sample Complexity Bounds for Active Ranking from Multi-wise Comparisons
Authors: Wenbo Ren, Jia Liu, Ness Shroff
NeurIPS 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also conduct numerical simulations to confirm our theoretical results. |
| Researcher Affiliation | Academia | Wenbo Ren Dept. Computer Science & Engineering The Ohio State University EMAIL Jia Liu Dept. Electrical & Computer Engineering The Ohio State University EMAIL Ness B. Shroff Dept. Electrical & Computer Engineering and Computer Science & Engineering The Ohio State University EMAIL |
| Pseudocode | Yes | Algorithm 1 Multi-wise Quick-Select(S, m, k) (MQSelect). |
| Open Source Code | Yes | The codes can be found in our Git Hub repo.6 https://github.com/Wenbo Ren/Multi-wise-Ranking.git |
| Open Datasets | No | The paper discusses "a set of n items" and for numerical results states that "all points are averaged over 100 independent trials with random true rankings." It does not provide access information for a publicly available or open dataset. |
| Dataset Splits | No | The paper describes numerical simulations with varying parameters (n, k, m, δ) and using "random true rankings" for trials, but it does not provide specific training/test/validation dataset splits or mention any predefined splits. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions that code is available in a GitHub repository but does not provide specific ancillary software details, such as library or solver names with version numbers. |
| Experiment Setup | Yes | In Figure 1 (a), we set n = 1000 and k = {1, 10, 100, 500}, and vary the value of m. In Figure 1 (b), we set n = 1000 and m = {2, 10, 100, 500}, and vary the value of k. In all figures, n = 1000 (except (f)), δ = 0.01 (if applicable), and all points are averaged over 100 independent trials with random true rankings. |