Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Byzantine Spectral Ranking
Authors: Arnhav Datar, Arun Rajkumar, John Augustine
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we confirm our theoretical bounds with experiments conducted on synthetic and real data. |
| Researcher Affiliation | Academia | Arnhav Datar Indian Institute of Technology, Madras EMAIL Arun Rajkumar Indian Institute of Technology, Madras EMAIL John Augustine Indian Institute of Technology, Madras EMAIL |
| Pseudocode | Yes | Algorithm 1 Rank-Centrality Algorithm 2 Byzantine Spectral Ranking Algorithm 3 Fast Byzantine Spectral Ranking |
| Open Source Code | No | The paper states 'Yes' in the checklist for including code, data, and instructions needed to reproduce main experimental results, but does not provide a direct link or explicit statement about open-sourcing its *own* code within the main body of the paper. |
| Open Datasets | Yes | We consider the Sushi dataset comprising 5000 voters ordering the 10 sushis from most preferred to least preferred. |
| Dataset Splits | No | The paper does not explicitly provide specific train/validation/test dataset splits, percentages, or methods for partitioning the data, for either the synthetic or real datasets. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or cloud instances) used for running its experiments. |
| Software Dependencies | No | The paper does not provide a reproducible description of ancillary software with specific version numbers for key components or libraries. |
| Experiment Setup | Yes | We consider the following parameters in our testing (1) n = 200 (2) k = 100 (3) p = 20 log n/n and (4) wi = Uniform(1, 100). We applied the FBSR Algorithm with the following modifications considering the smaller k and n values: (1) setting 5δ = 1 + psize(Bbucket) and (2) setting max_out as k/20. |