Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Unifying Knowledge Base Completion with PU Learning to Mitigate the Observation Bias
Authors: Jonas Schouterden, Jessa Bekker, Jesse Davis, Hendrik Blockeel4137-4145
AAAI 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We aim to empirically answer the following research questions: can we effectively account for observation biases (i.e., obtain more accurate confidence estimates) using the newly proposed propensity-based estimators, (Q1) when the propensities are known, (Q2) when propensities are guessed ( noisy propensities), (Q3) even when the PCA assumption holds? |
| Researcher Affiliation | Academia | KU Leuven, Department of Computer Science, B-3000 Leuven, Belgium Leuven.AI KU Leuven Institute for AI, B-3000 Leuven, Belgium EMAIL |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our source code is publicly available.5 5https://github.com/ML-KULeuven/KBC-as-PU-Learning |
| Open Datasets | Yes | Our I is the popular KBC benchmark dataset Yago3-10 (Mahdisoltani, Biega, and Suchanek 2015). |
| Dataset Splits | No | The paper does not specify explicit training, validation, and test dataset splits with percentages or sample counts to reproduce the data partitioning. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running the experiments. |
| Software Dependencies | No | The paper mentions that 'Rules predicting any p P are mined from I with AMIE (Galárraga et al. 2015) with its default settings', but does not provide specific version numbers for software dependencies like programming languages or libraries. |
| Experiment Setup | Yes | Rules predicting any p P are mined from I with AMIE (Galárraga et al. 2015) with its default settings and a minimum CWA(R) 0.1. [...] Propensity scores are required to calculate IPW(-PCA)(R). We use correct propensity scores e( ) for the idealized scenarios in Q1 and Q3 and noisy versions ˆe for Q2. More details about the exact setup can be found in Appendix C. |