Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Interacting with Explanations through Critiquing
Authors: Diego Antognini, Claudiu Musat, Boi Faltings
IJCAI 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results show that our system achieves good performance in adapting to the preferences expressed in multi-step critiquing and generates consistent explanations. We evaluate our model using two real-world recommendation datasets. |
| Researcher Affiliation | Collaboration | 1 Ecole Polytechnique F ed erale de Lausanne, Switzerland 2Swisscom, Switzerland {diego.antognini, boi.faltings}@epfl.ch, EMAIL |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing its source code or a direct link to a code repository for the described methodology. A link to appendices is provided, but it's not a code repository. |
| Open Datasets | Yes | We evaluate the quantitative performance of TRECS using two real-world, publicly available datasets: Beer Advocate [Mc Auley and Leskovec, 2013] and Hotel Rec [Antognini and Faltings, 2020]. |
| Dataset Splits | Yes | We keep the first 80% of interactions per user as the training data, leaving the remaining 20% for validation and testing. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments (e.g., GPU/CPU models, memory). |
| Software Dependencies | No | The paper mentions using "Adam" as an optimizer but does not provide specific version numbers for any software components, libraries, or programming languages used. |
| Experiment Setup | Yes | We set the embedding and attention dimension to 256 and to 1024 for the feed-forward network. The encoder and decoder consist of two layers of Transformer with 4 attention heads. We use a batch size of 128, dropout of 0.1, and Adam with learning rate 0.001. For critiquing, we choose a threshold and decay coefficient T = 0.015, ζ = 0.9 and T = 0.01, ζ = 0.975 for hotel and beer reviews. |