Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Relation-aware Graph Attention Model with Adaptive Self-adversarial Training
Authors: Xiao Qin, Nasrullah Sheikh, Berthold Reinwald, Lingfei Wu9368-9376
AAAI 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental evaluation demonstrates that Rel GNN optimized by ASA for relationship prediction improves state-of-the-art performance across established benchmarks as well as on a real industrial dataset. We evaluate Rel GNN and ASA negative sampler by an ablation study and a position study which compares our method against state-of-the-art approaches in relationship prediction task on inductive benchmarks as well as a real industrial dataset. 4 Experimental Evaluation This section summarizes our experimental setup and reports, and analyzes the measurements. Dataset. We use two established heterogeneous graph datasets, namely Amazon and Youtube (Cen et al. 2019) which come with standard validation and test sets. |
| Researcher Affiliation | Industry | Xiao Qin1 , Nasrullah Sheikh1 , Berthold Reinwald1, Lingfei Wu2 1IBM Almaden Research Center, 2IBM Thomas J. Watson Research Center EMAIL, EMAIL |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statement about releasing source code or a link to a code repository. |
| Open Datasets | Yes | Dataset. We use two established heterogeneous graph datasets, namely Amazon and Youtube (Cen et al. 2019) which come with standard validation and test sets. |
| Dataset Splits | Yes | Dataset. We use two established heterogeneous graph datasets, namely Amazon and Youtube (Cen et al. 2019) which come with standard validation and test sets. Basic statistics of these datasets are reported in Table 1. Table 1: Statistics of the datasets. ... Relations (train/valid/test) Amazon (Cen et al. 2019) ... 85/5/10 Youtube (Cen et al. 2019) ... 85/5/10 Company ... 80/10/10 |
| Hardware Specification | No | The paper does not specify the hardware used for running the experiments (e.g., CPU, GPU models, or cloud resources). |
| Software Dependencies | No | The paper mentions general software components like 'RNN', 'MLP', and 'geo-location API' but does not specify any particular software libraries or their version numbers required to reproduce the experiments. |
| Experiment Setup | Yes | We also introduce a margin µ as a hyperparameter to further control the hardness the higher the µ the easier the case. ... We use MRR and Hit@k to evaluate the methods where the perfect scores can be achieved by scoring the true relationships higher than all the respective negative samples. Table 3 reports the performance of Rel GNN trained by different negative sampling techniques. We vary the pool size for NSCaching and ASA (constant µ = 0.1). |