Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Progressive Comparison for Ranking Estimation
Authors: Ryusuke Takahama, Toshihiro Kamishima, Hisashi Kashima
IJCAI 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The results of experiments using a synthetic dataset and two real datasets demonstrate that the progressive comparison method achieves high estimation accuracy with a smaller number of evaluations than the standard pairwise comparison method, and that the active learning methods further reduce the number of evaluations as compared with a random sampling method. |
| Researcher Affiliation | Academia | 1Kyoto University 2JST, ERATO, Kawarabayashi Large Graph Project 3National Institute of Advanced Industrial Science and Technology (AIST) |
| Pseudocode | No | No pseudocode or clearly labeled algorithm block found. |
| Open Source Code | No | The paper does not provide a link or explicit statement about the availability of open-source code for the described methodology. |
| Open Datasets | Yes | The dataset is available at http://goo.gl/6MS9MK. |
| Dataset Splits | No | The paper does not explicitly provide training/test/validation dataset splits. It describes initialization parameters and comparing estimated ranking lists with true ones, but no specific data partitioning. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers. |
| Experiment Setup | Yes | Throughout the experiments, the parameters are initialized ยตi = 1, 500 and ฯ2i = 1472 = 21, 609 by following the settings of the glicko update equations [Glickman, 1999]. |