Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Efficient Sequential Model-Based Fault-Localization with Partial Diagnoses
Authors: Kostyantyn Shchekotykhin, Thomas Schmitz, Dietmar Jannach
IJCAI 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | An experimental evaluation on different benchmark problems shows that our sequential diagnosis approach needs considerably less computation time when compared with an existing domain-independent approach. |
| Researcher Affiliation | Academia | Kostyantyn Shchekotykhin,1 Thomas Schmitz,2 and Dietmar Jannach2 1Alpen-Adria University Klagenfurt, Austria e-mail: EMAIL 2TU Dortmund, Germany e-mail: {๏ฌrstname.lastname}@tu-dortmund.de |
| Pseudocode | Yes | Algorithm 1 summarizes our approach, which in contrast to previous works can operate on the basis of partial diagnoses. |
| Open Source Code | No | The paper states that 'The algorithms were implemented in Java' but does not provide any link, repository, or explicit statement of code availability for the methodology described. |
| Open Datasets | Yes | We evaluated our method on two sets of benchmark problems: (a) the ontologies of the OAEI Conference benchmark as used in [Shchekotykhin et al., 2014], (b) the systems of the DX Competition (DXC) 2011 Synthetic Track. |
| Dataset Splits | No | The paper describes running experiments multiple times with randomly selected preferred diagnoses or scenarios, but it does not specify explicit training, validation, and testing dataset splits. |
| Hardware Specification | No | All tests were performed on a modern laptop computer. This description is too vague and does not provide specific hardware details (e.g., CPU, GPU, memory). |
| Software Dependencies | No | The algorithms were implemented in Java. Choco was used as a constraint solver and Hermi T as Description Logic reasoner. However, specific version numbers for Java, Choco, or Hermi T are not provided. |
| Experiment Setup | Yes | For both strategies, we set the number of diagnoses n that are used to determine the optimal query to 9 as done in [Shchekotykhin et al., 2014], and used the best-performing Entropy strategy for query selection (GETQUERY). |