Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
ReFactor GNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective
Authors: Yihong Chen, Pushkar Mishra, Luca Franceschi, Pasquale Minervini, Pontus Lars Erik Saito Stenetorp, Sebastian Riedel
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform experiments to answer the following questions regarding REFACTOR GNNS: Q1. REFACTOR GNNS are derived from a message-passing reformulation of FMs: do they also inherit FMs predictive accuracy in transductive KGC tasks? (Section 5.1) Q2. REFACTOR GNNS inductivise FMs. Are they more statistically accurate than other GNN baselines in inductive KGC tasks? (Section 5.2) Q3. The term n[v] involves nodes that are not in the 1-hop neighbourhood. Is such augmented message passing [41] necessary for good KGC performance? (Section 5.3) For transductive experiments, we used three well-established KGC datasets: UMLS [14], CoDEx-S [31], and FB15K237 [38]. |
| Researcher Affiliation | Collaboration | Yihong Chenàá Pushkar Mishraá Luca Franceschiâ Pasquale Minerviniàæ* Pontus Stenetorpà Sebastian Riedelàá àUCL Centre for Artificial Intelligence, London, United Kingdom áMeta AI, London, United Kingdom âAmazon Web Services, Berlin, Germany æSchool of Informatics, University of Edinburgh, Edinburgh, United Kingdom |
| Pseudocode | No | The paper describes methods using mathematical equations and textual explanations but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The implementation will be available https://github.com/yihong-chen/Re Factor GNN. |
| Open Datasets | Yes | For transductive experiments, we used three well-established KGC datasets: UMLS [14], CoDEx-S [31], and FB15K237 [38]. For inductive experiments, we used the inductive KGC benchmarks introduced by GraIL [37], which include 12 pairs of knowledge graphs: (FB15K237_vi, FB15K237_vi_ind), (WN18RR_vi, WN18RR_vi_ind), and (NELL_vi, NELL_vi_ind), where i ∈ {1, 2, 3, 4}, and (_vi, _vi_ind) represents a pair of graphs with a shared relation vocabulary and non-overlapping entities. |
| Dataset Splits | Yes | We performed a grid search over the other hyper-parameters and selected the best configuration based on the validation MRR. |
| Hardware Specification | No | The paper mentions 'high utilisation of GPUs' but does not provide specific hardware details such as GPU models, CPU types, or memory amounts used for experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., libraries, frameworks, or languages with their versions). |
| Experiment Setup | Yes | We used a hidden size of 768 for the node representations. All the models are trained using [128, 512] in-batch negative samples and one global negative node for each positive link. We performed a grid search over the other hyper-parameters and selected the best configuration based on the validation MRR. |