Recursive Inversion Models for Permutations
Authors: Christopher Meek, Marina Meila
NeurIPS 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We performed experiments on synthetic data and real-world data sets. In our synthetic experiments we found that our approach was typically able to identify both the structure and parameters of the generative model. More specifically, we ran extensive experiments with n = 16 and n = 33, choosing the model structures to have varying degrees of balance, and choosing the parameters randomly chosen with exp( θi) between 0.4 and 0.9. We then used these RIMs to generate datasets containing varying numbers of permutations to investigate whether the true model could be recovered. We found that all models were recoverable with high probability when using between 200-1000 SASEARCH iterations. We did find that the identification of the correct tree structure in its entirety typically required a large sample size. We note that failures to identify the correct structure were typically due to the fact that alternative structures had higher likelihood than the generating structure in a particular sample rather than a failure of the search algorithm. While our experiments had at most n = 33 this was not due to the running time of the algorithms. For instance, STRUCTBYDP ran in a few seconds for domains with 33 items. For the smaller domains and for the real-world data below, the whole search with hundreds of accepted proposals typically ran in less than three minutes. |
| Researcher Affiliation | Collaboration | Christopher Meek Microsoft Research Redmond, Washington 98052 meek@microsoft.com Marina Meil a University of Washington Seattle, Washington 98195 mmp@stat.washington.edu |
| Pseudocode | Yes | Algorithm 1 Algorithm CANONICALPERMUTATION (...) Algorithm 2 Algorithm STRUCTBYDP (...) Algorithm 3 Algorithm SASEARCH |
| Open Source Code | No | The paper does not provide an explicit statement or link to open-source code for the Recursive Inversion Models or the SASEARCH algorithm described in the paper. It mentions that code for a compared method (Huang & Guestrin) was not available to them. |
| Open Datasets | Yes | We use the roughly 2500 fully ranked ballots from the election. See Gormley & Murphy (2007) for more details about the dataset. The second dataset consists of 5,000 permutations of 10 different types of sushi where the permutation captures preferences about sushi (Kamisha, 2003). |
| Dataset Splits | No | The paper specifies "Train/test set split was 90/2400, respectively 300/4700", indicating training and testing splits, but does not explicitly mention a "validation" set or its split. |
| Hardware Specification | No | The paper mentions the running time of the algorithms, stating "STRUCTBYDP ran in a few seconds for domains with 33 items" and "the whole search with hundreds of accepted proposals typically ran in less than three minutes." However, it does not provide any specific hardware details such as CPU, GPU models, or memory specifications. |
| Software Dependencies | No | The paper does not explicitly list any specific software components or libraries with their version numbers that were used for the experiments. |
| Experiment Setup | Yes | We ran extensive experiments with n = 16 and n = 33, choosing the model structures to have varying degrees of balance, and choosing the parameters randomly chosen with exp( θi) between 0.4 and 0.9. (...) Finally, we search over both structures and orderings with SASEARCH, with 150 (100) iterations for Meath (sushi) at temperature 0.02. |