Ranking via Robust Binary Classification
Authors: Hyokun Yun, Parameswaran Raman, S. Vishwanathan
NeurIPS 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | It shows competitive performance on standard benchmark datasets against a number of other representative algorithms in the literature. We show that Ro Bi Rank can be efficiently parallelized across a large number of machines; for a task that requires 386, 133 49, 824, 519 pairwise interactions between items to be ranked, Ro Bi Rank finds solutions that are of dramatically higher quality than that can be found by a state-of-the-art competitor algorithm, given the same amount of wall-clock time for computation. |
| Researcher Affiliation | Collaboration | Hyokun Yun Amazon Seattle, WA 98109 yunhyoku@amazon.com Parameswaran Raman, S. V. N. Vishwanathan Department of Computer Science University of California Santa Cruz, CA 95064 {params,vishy}@ucsc.edu |
| Pseudocode | Yes | Pseudo-code can be found in Algorithm 1 in Appendix C. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described. It mentions that the code for Weston et al. [24] was not available, implying they implemented it, but makes no statement about releasing their own implementation. |
| Open Datasets | Yes | We use three sources of datasets: LETOR 3.0 [8] , LETOR 4.02 and YAHOO LTRC [20], which are standard benchmarks for ranking... In this subsection we use the Million Song Dataset (MSD) [3], which consists of 1,129,318 users, 386,133 songs, and 49,824,519 records of a user x playing a song y in the training dataset. |
| Dataset Splits | Yes | Each dataset consists of five folds; we consider the first fold, and use the training, validation, and test splits provided. We train with different values of regularization parameter, and select one with the best NDCG on the validation dataset. |
| Hardware Specification | No | The paper mentions using 'Amazon Web Services' and 'Texas Advanced Computing Center for infrastructure and support for experiments', but does not provide specific hardware details such as GPU models, CPU models, or memory specifications. |
| Software Dependencies | No | The paper mentions using 'L-BFGS algorithm provided by the Toolkit for Advanced Optimization (TAO)' but does not provide a specific version number for TAO or any other software dependencies. |
| Experiment Setup | No | The paper generally mentions training with 'different values of regularization parameter' and 'grid-search over the step size parameter', but it does not provide concrete numerical values for these or other hyperparameters, nor does it detail specific training configurations like batch size or number of epochs. |