Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues
Authors: Nihar Shah, Sivaraman Balakrishnan, Aditya Guntuboyina, Martin Wainwright
ICML 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We complement our theoretical results with thorough numerical simulations. The simulations in this section add to the simulation results of Section 2.4 (Figure 1) demonstrating a large class of matrices in the SST class that cannot be represented by any parametric class. We investigate the performance of the soft-SVT estimator (Section 3.2) and the maximum likelihood estimator under the Thurstone model (Section 2.3). Figure 2 depicts the results of the simulations based on observations of the entire matrix Y. |
| Researcher Affiliation | Academia | Nihar B. Shah NIHAR@EECS.BERKELEY.EDU Sivaraman Balakrishnan] SIVA@STAT.CMU.EDU Adityanand Guntuboyina ADITYA@STAT.BERKELEY.EDU Martin J. Wainwright WAINWRIG@BERKELEY.EDU Dept. of EECS, Dept. of Statistics, University of California, Berkeley ] Dept. of Statistics, Carnegie Mellon University |
| Pseudocode | No | The paper describes algorithms (e.g., SVT estimator, two-step estimator) in prose, detailing steps and mathematical formulations, but it does not include any dedicated pseudocode blocks, figures, or algorithm listings. |
| Open Source Code | No | The paper does not contain any explicit statement about releasing source code for the methodology or a link to a code repository. |
| Open Datasets | No | In our simulations, we generate the ground truth M in the following five ways: Uniform, Thurstone, Bradley-Terry-Luce (BTL), High SNR, Independent bands. |
| Dataset Splits | No | The paper describes how the ground truth data (M) for simulations is generated and then how estimators are applied to observations (Y) from this M. However, it does not specify any training, validation, or test dataset splits for its own experimental setup or model training beyond stating that observations are made of the entire matrix Y. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory) used to run the simulations or experiments. |
| Software Dependencies | No | The paper mentions statistical models and algorithms but does not specify any software names with version numbers (e.g., Python 3.x, PyTorch 1.x, specific solvers) that would be needed for reproducibility. |
| Experiment Setup | No | The paper describes the data generation process for simulations and the estimators used (SVT, Thurstone MLE), including a threshold parameter for SVT. However, it lacks concrete details typical of an experimental setup, such as learning rates, batch sizes, specific optimizers, number of training epochs, or other hyperparameter values for the algorithms when applied in the simulations. It describes the algorithms themselves rather than their specific experimental configurations. |