Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Preference-Based Dynamic Ranking Structure Recognition
Authors: Nan Lu, Jian Shi, Xinyu Tian
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on both synthetic and real-world datasets demonstrate the practical utility and interpretability of our approach. 4 Computational Experiments 5 Empirical Analysis: Ranking Structure Recognition of NBA Teams |
| Researcher Affiliation | Academia | Nan Lu1,2 EMAIL Jian Shi1,2, EMAIL Xin-Yu Tian3 EMAIL 1State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China 2School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, China 3School of Statistics, University of Minnesota, Minneapolis, USA |
| Pseudocode | Yes | We can address the combinatorial problem using Algorithm 1, which provides an efficient method to estimate R = {bsi}i [ b J]. Algorithm 1 Structure Change Detection Require: Observed data Y, tuning parameters γ1, γ2. Ensure: Change points estimation R. |
| Open Source Code | Yes | Corresponding author. Code available at https://github.com/nanlu99/RankStruct. The codes for the proposed algorithm and experiments will also be made publicly available upon publication. |
| Open Datasets | Yes | Experiments on both synthetic and real-world datasets demonstrate the practical utility and interpretability of our approach. We analyze NBA regular season data from the 2014-2015 season to the 2018-2019 season2. 2https://www.nba.com/games |
| Dataset Splits | Yes | We employ the widely-used 10-fold cross-validation for the choice of tuning parameters γ1 and γ2. |
| Hardware Specification | Yes | All experiments are conducted on a machine with an 11th Gen Intel(R) Core(TM) i5-1135G7 CPU and 16GB RAM. |
| Software Dependencies | No | We utilize R pacakge sparsegl [34] for analysis. |
| Experiment Setup | Yes | For the experiments in Section 4.1, we consider two settings to evaluate our methods, as illustrated in Figure 4. We set T = 1, B = 3, and assign the number of items in each group as 3:3:4. ... The point ϵ used for order estimation is 0.001. We set h = 0.05, m = 30 and vary n and M. We repeat each setting 500 times and use the extended BIC (EBIC) criterion [35, 22] to choose the tuning parameter λ. ... In Section B.2, 'Experiment settings for group changes recognition' it states: 'We set h = 0.02 and denote the observation times within each phase as M. The experiment is repeated 500 times. We employ the widely-used 10-fold cross-validation for the choice of tuning parameters γ1 and γ2.' |