Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Empirical Analysis of Foundational Distinctions in Linked Open Data
Authors: Luigi Asprino, Valerio Basile, Paolo Ciancarini, Valentina Presutti
IJCAI 2018 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We report on a set of experiments based on machine learning and crowdsourcing that show promising results. In this study, we perform a set of experiments to assess whether machines can learn to perform foundational distinctions, and if they match common sense. |
| Researcher Affiliation | Academia | 1 STLab, ISTC-CNR, Rome, Italy 2 University of Bologna, Bologna, Italy 3 University of Turin, Turin, Italy |
| Pseudocode | No | The paper describes the methods in prose and does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The resulting datasets include a sample of annotated DBpedia entities and are available online 17. 17https://github.com/fdistinctions/ijcai18 |
| Open Datasets | Yes | The resulting datasets include a sample of annotated DBpedia entities and are available online 17. 17https://github.com/fdistinctions/ijcai18. We use DBpedia (release 2016-10) in our study as most LOD datasets link to it. |
| Dataset Splits | Yes | We used a 10-fold cross validation strategy using the reference datasets (cf. Section 4). Before training the classifiers, the datasets were adjusted in order to balance the samples of the two classes. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU models, or cloud instance specifications used for running the experiments. |
| Software Dependencies | No | We have used WEKA16 for their implementation. (Footnote 16: https://www.cs.waikato.ac.nz/ml/weka/) - The paper mentions WEKA but does not specify a version number for it or any other software libraries used. |
| Experiment Setup | No | The paper describes the features used for the machine learning models and the cross-validation strategy, but it does not provide specific hyperparameters (e.g., learning rate, batch size, epochs) or detailed system-level training settings for the algorithms. |