Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Impartial Selection with Predictions
Authors: Javier Cembrano, Felix Fischer, Max Klimm
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study the selection of agents based on mutual nominations, a theoretical problem with many applications from committee selection to AI alignment. ... The paper does not include experiments. All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. All assumptions should be clearly stated or referenced in the statement of any theorems. |
| Researcher Affiliation | Academia | Javier Cembrano Department of Algorithms and Complexity Max-Planck-Institut für Informatik Saarbrücken, Germany Department of Industrial Engineering Universidad de Chile Santiago, Chile EMAIL Felix Fischer School of Mathematical Sciences Queen Mary University of London London, UK EMAIL Max Klimm Institute of Mathematics Technische Universität Berlin Berlin, Germany EMAIL |
| Pseudocode | Yes | Algorithm 1 Permutation mechanism Pm(G, S, x) ... Algorithm 2 ρ-permutation mechanism Pmρ(ˆı, G) ... Algorithm 3 Fixed bidirectional permutation mechanism, Pmbi( ˆS, G) ... Algorithm 4 ρ-partition mechanism Ptρ( ˆS, G) |
| Open Source Code | No | Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [NA] Justification: The paper does not include experiments requiring code. |
| Open Datasets | No | Question: Does the paper provide CONCRETE ACCESS INFORMATION (specific link, DOI, repository name, formal citation with authors/year, or reference to established benchmark datasets) for a publicly available or open dataset? Answer: [NA] Justification: The paper does not include experiments. |
| Dataset Splits | No | Question: Does the paper explicitly provide training/test/validation dataset splits needed to reproduce the experiment? Answer: [NA] Justification: The paper does not include experiments. |
| Hardware Specification | No | Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [NA] Justification: The paper does not include experiments. |
| Software Dependencies | No | The paper does not include experiments requiring code. Thus, no specific software dependencies with version numbers are mentioned. |
| Experiment Setup | No | Question: Does the paper explicitly provide details about the experimental setup, especially hyperparameters or system-level training settings? Answer: [NA] Justification: The paper does not include experiments. |