Constructive Preference Elicitation by Setwise Max-Margin Learning
Authors: Stefano Teso, Andrea Passerini, Paolo Viappiani
IJCAI 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the effectiveness of our approach by testing our elicitation method in both synthetic and realworld problems, and comparing it to state-of-the-art methods. In Figure 2 we report solution quality and timing values for increasing number of collected user responses, for the different competitors on each of the four different utility vector distributions and datasets r = 3 and r = 4. |
| Researcher Affiliation | Academia | Stefano Teso University of Trento Trento, Italy teso@disi.unitn.it Andrea Passerini University of Trento Trento, Italy passerini@disi.unitn.it Paolo Viappiani Sorbonne Universit es UPMC Univ Paris 06 CNRS, LIP6 UMR 7606 Paris, France paolo.viappiani@lip6.fr |
| Pseudocode | Yes | Algorithm 1 The SETMARGIN algorithm. |
| Open Source Code | Yes | Both the SETMARGIN source code and the full experimental setup are available at https://github.com/stefanoteso/setmargin. |
| Open Datasets | No | We developed synthetic datasets with r attributes, for increasing values of r. We developed a constructive version of the PC dataset used in [Guo and Sanner, 2010]: instead of explicitly enumerating all possible PC items, we defined the set of feasible configurations with MILP constraints. The paper does not provide links or citations to publicly available datasets for download. |
| Dataset Splits | Yes | In all experiments SETMARGIN uses an internal 5-fold cross-validation procedure to update the hyperparameters , β, and γ after every 5 iterations. |
| Hardware Specification | Yes | All experiments were run on a 2.8 GHz Intel Xeon CPU with 8 cores and 32 Gi B of RAM. |
| Software Dependencies | Yes | We implemented the SETMARGIN algorithm using Python, leveraging Gurobi 6.5.0 for solving the core MILP problem. |
| Experiment Setup | Yes | The parameters λ1 and λ2 were set to one for all simulations, as in [Guo and Sanner, 2010]. In all experiments SETMARGIN uses an internal 5-fold cross-validation procedure to update the hyperparameters , β, and γ after every 5 iterations. The hyperparameters are chosen as to minimize the ranking loss over the user responses collected so far. is taken in {20, 10, 5, 1}, while β and γ are taken in {10, 1, 0.1, 0.001}. We set a maximum budget of 100 iterations for all methods for simplicity. For all algorithms, one iteration corresponds to a single pairwise query (we used SETMARGIN with k = 2). |