Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Coactive Critiquing: Elicitation of Preferences and Features
Authors: Stefano Teso, Paolo Dragone, Andrea Passerini
AAAI 2017 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical ๏ฌndings highlight the promise of Coactive Critiquing in a synthetic and a realistic preference elicitation problem, highlighting its ability in offering a reasonable trade-off between the quality of the recommendations and the cognitive effort expected from the user. |
| Researcher Affiliation | Academia | Stefano Teso EMAIL University of Trento Via Sommarive 9, Povo Trento, Italy Paolo Dragone EMAIL University of Trento TIM-SKIL Via Sommarive 9, Povo Trento, Italy Andrea Passerini EMAIL University of Trento Via Sommarive 9, Povo Trento, Italy |
| Pseudocode | Yes | Algorithm 1 Pseudo-code of the Coactive Critiquing algorithm. Here ฯ1 is the initial feature space, and T is the maximum number of iterations. User interaction occurs inside the QUERYIMPROVEMENT and QUERYCRITIQUE procedures. |
| Open Source Code | Yes | The CC source code and the full experimental setup are available at: goo.gl/c TFOFq. |
| Open Datasets | No | The paper does not provide concrete access information (link, DOI, formal citation) for the datasets used beyond general descriptions. For example, for the 'Realistic Experiment', it mentions 'We collected a dataset including 10 cities and 15 possible activities from the Trentino Open data website: http://dati.trentino.it/.' While a website is provided, it's not a direct link to the specific dataset used, nor is it a formal citation with authors and year for a well-established dataset. |
| Dataset Splits | No | The paper does not specify exact training/validation/test split percentages or sample counts. It describes experiments but not how the data was partitioned for different phases. |
| Hardware Specification | Yes | All experiments were run on a 2.8 GHz Intel Xeon CPU with 8 cores and 32 Gi B of RAM. |
| Software Dependencies | Yes | Our implementation makes use of Mini Zinc (Nethercote et al. 2007) with the Gecode backend. |
| Experiment Setup | No | The paper describes the simulated user behavior for improvement and critiquing queries and outlines the setup for synthetic and realistic experiments (e.g., number of rectangles, trip length). However, it lacks specific hyperparameter values (like learning rate, batch size) or detailed system-level training settings which are typically part of a comprehensive experimental setup. |