Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Byzantine Spectral Ranking
Authors: Arnhav Datar, Arun Rajkumar, John Augustine
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we confirm our theoretical bounds with experiments conducted on synthetic and real data. |
| Researcher Affiliation | Academia | Arnhav Datar Indian Institute of Technology, Madras EMAIL Arun Rajkumar Indian Institute of Technology, Madras EMAIL John Augustine Indian Institute of Technology, Madras EMAIL |
| Pseudocode | Yes | Algorithm 1 Rank-Centrality Algorithm 2 Byzantine Spectral Ranking Algorithm 3 Fast Byzantine Spectral Ranking |
| Open Source Code | No | The paper states 'Yes' in the checklist for including code, data, and instructions needed to reproduce main experimental results, but does not provide a direct link or explicit statement about open-sourcing its *own* code within the main body of the paper. |
| Open Datasets | Yes | We consider the Sushi dataset comprising 5000 voters ordering the 10 sushis from most preferred to least preferred. |
| Dataset Splits | No | The paper does not explicitly provide specific train/validation/test dataset splits, percentages, or methods for partitioning the data, for either the synthetic or real datasets. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or cloud instances) used for running its experiments. |
| Software Dependencies | No | The paper does not provide a reproducible description of ancillary software with specific version numbers for key components or libraries. |
| Experiment Setup | Yes | We consider the following parameters in our testing (1) n = 200 (2) k = 100 (3) p = 20 log n/n and (4) wi = Uniform(1, 100). We applied the FBSR Algorithm with the following modifications considering the smaller k and n values: (1) setting 5δ = 1 + psize(Bbucket) and (2) setting max_out as k/20. |