NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs
Authors: Mikhail Galkin, Etienne Denis, Jiapeng Wu, William L. Hamilton
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that Node Piece performs competitively in node classification, link prediction, and relation prediction tasks while retaining less than 10% of explicit nodes in a graph as anchors and often having 10x fewer parameters. |
| Researcher Affiliation | Academia | Mikhail Galkin, Etienne Denis, Jiapeng Wu and William L. Hamilton Mila, Mc Gill University Montreal, Canada {mikhail.galkin,deniseti,jiapeng.wu,hamilton}@mila.quebec |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled "Pseudocode" or "Algorithm". |
| Open Source Code | Yes | The code is available on Git Hub: https://github.com/migalkin/Node Piece |
| Open Datasets | Yes | All used datasets are available under open licenses. Details on the datasets for transductive link prediction, out-of-sample link prediction, relation prediction and node classification are collected in Table 8. The inductive link prediction benchmark introduced by Teru et al. (2020) includes 3 graphs, FB15k-237, WN18RR, and NELL-995 |
| Dataset Splits | Yes | Details on the datasets for transductive link prediction, out-of-sample link prediction, relation prediction and node classification are collected in Table 8. The inductive link prediction benchmark introduced by Teru et al. (2020) includes 3 graphs, FB15k-237, WN18RR, and NELL-995, each has 4 different splits that vary in the number of unique relations, number of nodes and triples at training and inference time. Full dataset statistics is provided in Table 9. |
| Hardware Specification | Yes | We ran experiments on a machine with one RTX 8000 GPU and 64 GB RAM. The OGB Wiki KG 2 experiments were executed on a single Tesla V100 16 GB VRAM and 64 GB RAM. |
| Software Dependencies | Yes | Node Piece is implemented in Python using igraph library (licensed under GNU GPL 2) for computing centrality measures and perform basic tokenization. Downstream tasks employ Node Piece in conjunction with Py Torch (Paszke et al., 2019) (BSD-style license), Py KEEN (Ali et al., 2021) (MIT License), and Py Torch-Geometric (Fey & Lenssen, 2019) (MIT License). |
| Experiment Setup | Yes | Table 10: Node Piece hyperparameters for transductive link prediction experiments. Parameter FB15k-237 WN18RR Co DEx-L YAGO 3-10 OGB Wiki KG 2 # Anchors, |A| 1000 500 7000 10000 20000 # Anchors per node, k 20 50 20 20 20 Relational context, m 15 4 6 5 12 Vocabulary dim, d 200 200 200 200 200 Batch size 512 512 256 512 512 Learning rate 0.0005 0.0005 0.0005 0.00025 0.0001 Epochs 400 600 120 600 300k (steps) Encoder type MLP MLP MLP MLP MLP Encoder dim 400 400 400 400 400 Encoder layers 2 2 2 2 2 Encoder dropout 0.1 0.1 0.1 0.1 0.1 Loss function BCE NSSAL NSSAL NSSAL NSSAL Margin 15 50 50 # Negative samples 20 10 128 Label smoothing 0.4 0.3 - |