Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Enhancing Statement Evaluation in Argumentation via Multi-labelling Systems
Authors: Pietro Baroni, Regis Riveret
JAIR 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | we introduce multi-labelling systems, a generic formalism devoted to represent reasoning processes consisting of a sequence of labelling stages. In this context, two families of multi-labelling systems, called argument-focused and statement-focused approach, are identified and compared. Then they are shown to be able to encompass several prominent literature proposals as special cases, thereby enabling a systematic comparison evidencing their merits and limits. Further, we show that the proposed model supports tunability of statement justification by specifying a few alternative statement justification labellings, and we illustrate how they can be seamlessly integrated into different formalisms. The main contribution lies in a general framework, lacking in the previous literature, encompassing phases of argument and statement labellings in a unitary context. The formalism supports the identification of the two main classes of MLSs for argumentation mentioned above, of which we provide, as a further contribution, a formal comparison of expressiveness. |
| Researcher Affiliation | Academia | Pietro Baroni EMAIL DII, University of Brescia Brescia, Italy Regis Riveret EMAIL Data61, CSIRO Brisbane, Australia |
| Pseudocode | No | The paper defines abstract models, functions, and properties using formal mathematical notation and textual descriptions of processes (e.g., "the monolabelling Λ2-generator is such that...", "the polylabelling Λ2-generator is such that..."). It does not include any blocks explicitly labeled as "Pseudocode" or "Algorithm". |
| Open Source Code | No | The paper does not contain any statements about releasing source code, nor does it provide links to any code repositories for the methodology described. |
| Open Datasets | No | Example 1.1 (Adapted from Baroni, Governatori, Lam, & Riveret, 2016a; Baroni, Governatori, & Riveret, 2016b). Suppose that Dr. Smith says to you: Given your clinical data I conclude you are affected by disease D1 . Suppose then that another equally competent physician Dr. Jones says to you: Given your clinical data I conclude you are not affected by disease D1 . Your view on the justification of the statements s1 = I am affected by disease D1 and s1 = I am not affected by disease D1 may become quite uncertain." This is a hypothetical example for illustration, not an actual open dataset. No other datasets are mentioned or linked. |
| Dataset Splits | No | The paper focuses on theoretical modeling and uses hypothetical examples for illustration. It does not involve empirical experiments with datasets, and therefore no dataset split information is provided. |
| Hardware Specification | No | The paper is theoretical in nature, proposing a formal framework and analyzing its properties. It does not describe any experiments that would require specific hardware, and thus no hardware specifications are provided. |
| Software Dependencies | No | The paper focuses on theoretical contributions and formal modeling of argumentation systems. It does not describe any software implementation of its proposed multi-labelling systems, and therefore does not list any software dependencies with specific version numbers. |
| Experiment Setup | No | The paper is a theoretical work that introduces a formal framework and analyzes its properties using definitions, propositions, and illustrative examples. It does not describe any empirical experiments, and consequently, no experimental setup details like hyperparameters or system-level training settings are provided. |