Learning Mixtures of Plackett-Luce Models from Structured Partial Orders
Authors: Zhibing Zhao, Lirong Xia
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conducted experiments on synthetic data to demonstrate the effectiveness of our algorithms. The data are generated as follows: (i) generate α, θ(1), and θ(2) uniformly at random and normalize s.t. Pm i=1 θ(r) i = 1 for r = 1, 2; (ii) generate linear orders using k-PL-linear; (iii) choose φtop-l A , φl-way A , and φchoice-l A and sample partial orders from the generated linear orders. |
| Researcher Affiliation | Academia | Zhibing Zhao Department of Computer Science Rensselaer Polytechnic Institute Troy, NY 12180 zhaoz6@rpi.edu Lirong Xia Department of Computer Science Rensselaer Polytechnic Institute Troy, NY 12180 xial@cs.rpi.edu |
| Pseudocode | Yes | Algorithm 1 Algorithms for 2-PL-Φ. Input: Preference profile P with n partial orders. A set of preselected partial orders E. Output: Estimated parameter θ . Estimate φ using (2). For each E E, compute the frequency of E. Compute the output using (3). |
| Open Source Code | Yes | Code available at https://github.com/zhaozb08/Mix PL-SPO |
| Open Datasets | No | We conducted experiments on synthetic data to demonstrate the effectiveness of our algorithms. The data are generated as follows: (i) generate α, θ(1), and θ(2) uniformly at random and normalize s.t. Pm i=1 θ(r) i = 1 for r = 1, 2; (ii) generate linear orders using k-PL-linear; (iii) choose φtop-l A , φl-way A , and φchoice-l A and sample partial orders from the generated linear orders. |
| Dataset Splits | No | The algorithms are compared when the number of rankings varies (Figure 2). We have the following observations. When learning from partial orders only: ELSR-gibbs [16]" is much slower than other algorithms for large datasets. MSEs of all other algorithms converge towards zero as n increases. We can see top-2 and 2-way, partial" and choice, partial" converge slower than top-3". Ranked top-l orders are generally more informative for parameter estimation than other partial orders. However, as was reported in [34], it is much more time consuming for human to pick their ranked top alternative(s) from a large set of alternatives than fully rank a small set of alternatives, which means ranked top-l data are harder or more costly to collect. When learning from linear orders: our ranked top-2 and 2-way, linear" and choice-2, 3, 4, linear" outperform top-3 [33]" in terms of MSE (left of Figure 2), but only slightly slower than top-3 [33]" (Figure 2 right). (This section describes variations in data quantity and observation, but not standard train/validation/test splits.) |
| Hardware Specification | Yes | All algorithms were implemented with MATLAB1 on an Ubuntu Linux server with Intel Xeon E5 v3 CPUs each clocked at 3.50 GHz. |
| Software Dependencies | No | All algorithms were implemented with MATLAB1 on an Ubuntu Linux server with Intel Xeon E5 v3 CPUs each clocked at 3.50 GHz. (Only MATLAB is mentioned, but no version number or other software dependencies with versions are provided.) |
| Experiment Setup | Yes | The data are generated as follows: (i) generate α, θ(1), and θ(2) uniformly at random and normalize s.t. Pm i=1 θ(r) i = 1 for r = 1, 2; (ii) generate linear orders using k-PL-linear; (iii) choose φtop-l A , φl-way A , and φchoice-l A and sample partial orders from the generated linear orders. [...] For ELSR-Gibbs [16], we used the partial orders generated by choice-2, 3, 4". One linear extension was generated from each partial order and three EM iterations were run. All values were averaged over 2000 trials. |