Preference Completion: Large-scale Collaborative Ranking from Pairwise Comparisons
Authors: Dohyung Park, Joe Neeman, Jin Zhang, Sujay Sanghavi, Inderjit Dhillon
ICML 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We used the Movie Lens 100k dataset, which contains 100,000 ratings given by 943 users on 1682 movies. The ratings are given as integers from one to five, but we converted them into preference data by declaring that a user preferred one movie to another if they gave it a higher rating (if two movies received the same rating, we treated it as though the user did not provide a preference). Then we held out 20% of the data as a test set. We compare our algorithm with numerical rating based algorithms, CofiRank and LCR. We follow the standard setting that are used in the collaborative ranking literature (Weimer et al., 2007; Balakrishnan & Chopra, 2012; Volkovs & Zemel, 2012; Lee et al., 2014). For each user, we subsampled N ratings, used them for training, and took the rest of the ratings for test. The users with less than N + 10 ratings were dropped out. Table 2 compares Alt SVM with numerical rating based algorithms. |
| Researcher Affiliation | Academia | Dohyung Park DHPARK@UTEXAS.EDU Joe Neeman JOENEEMAN@GMAIL.COM Jin Zhang ZJ@UTEXAS.EDU Sujay Sanghavi SANGHAVI@MAIL.UTEXAS.EDU Inderjit S. Dhillon INDERJIT@CS.UTEXAS.EDU The University of Texas at Austin |
| Pseudocode | Yes | Algorithm 1 Alternating Support Vector Machine (Alt SVM) |
| Open Source Code | No | The competing algorithms are those with publicly available codes provided by the authors. |
| Open Datasets | Yes | We used the Movie Lens 100k dataset, which contains 100,000 ratings given by 943 users on 1682 movies. The datasets specified in Table 1. Table 1: Movie Lens1m, Movie Lens10m, Netflix |
| Dataset Splits | No | We used the Movie Lens 100k dataset, which contains 100,000 ratings given by 943 users on 1682 movies. ... Then we held out 20% of the data as a test set. For each user, we subsampled N ratings, used them for training, and took the rest of the ratings for test. |
| Hardware Specification | Yes | The experiments were run on a single 16-core machine in the Stampede Cluster at University of Texas. |
| Software Dependencies | No | We implemented the algorithm using the Open MP framework. |
| Experiment Setup | Yes | For each user, we subsampled N ratings, used them for training, and took the rest of the ratings for test. Both algorithms are set to estimate rank-100 matrices. We measured the time to achieve 10 5 tolerance on the binarized Movie Lens1m dataset. |