Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
On Ranking and Choice Models
Authors: Shivani Agarwal
IJCAI 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | As one example, Figure 2 shows the results obtained by applying our method to sports choice survey data in which 253 respondents each rated 7 sports on a scale of 1 5 based on how much they enjoyed each sport.5 As can be seen, the method accurately discovers two categories corresponding to individual and team sports, and identi๏ฌes a third category for jogging; |
| Researcher Affiliation | Academia | Shivani Agarwal Radcliffe Institute for Advanced Study, Harvard University, Cambridge, MA, USA Indian Institute of Science, Bangalore, India EMAIL |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | No | The paper does not provide any explicit statement or link indicating that the source code for the described methodologies is publicly available. |
| Open Datasets | No | The paper mentions 'sports choice survey data in which 253 respondents each rated 7 sports' used for an example, but does not provide a specific link, DOI, or formal citation for public access to this dataset. |
| Dataset Splits | No | The paper does not provide specific details on dataset splits (e.g., percentages, sample counts, or citations to predefined splits) for training, validation, or testing. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used to run experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers that would be needed to replicate the experiment. |
| Experiment Setup | No | The paper does not provide specific experimental setup details such as hyperparameter values, training configurations, or system-level settings. |