Bounded and Uniform Energy-based Out-of-distribution Detection for Graphs
Authors: Shenzhi Yang, Bin Liang, An Liu, Lin Gui, Xingkai Yao, Xiaofang Zhang
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that our approach dramatically improves the ability of GNNs to detect OOD data at the node level, e.g., in detecting OOD data induced by Structure Manipulation, the metric of FPR95 (lower is better) in scenarios without (with) OOD data exposure are reduced from the current SOTA by 28.4% (22.7%). |
| Researcher Affiliation | Academia | 1School of Computer Science and Technology, Soochow University, Suzhou, China 2Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Hong Kong, China 3Department of Informatics, King s College London, London, UK. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available via https://github.com/Shenzhi Yang2000/NODESAFE. |
| Open Datasets | Yes | We utilize five widely used realworld datasets in node classification tasks: Cora (Kipf & Welling, 2016), Citeseer (Kipf & Welling, 2016), Pubmed (Kipf & Welling, 2016), ogbn-Arxiv (Hu et al., 2020), and Twitch-Explicit (Wu et al., 2022). |
| Dataset Splits | Yes | We follow the partitioning standards of GNNSAFE (Wu et al., 2023) for dataset splits. For the Cora, Citeseer, and Pubmed datasets, we employ three approaches to generate OOD data: Structure Manipulation, Feature Interpolation, and Label Leave-out. For the Twitch dataset, OOD samples are derived from other graphs. We divide the samples based on temporal context for the Arxiv dataset. More specific details are elaborated in the Appendix B.1. For training/validation/testing, we adopt a semi-supervised learning approach inspired by (Kipf & Welling, 2016), utilizing the specified data splits on the in-distribution data. |
| Hardware Specification | Yes | All the experiments run with an NVIDIA 3090 with 24GB memory. |
| Software Dependencies | Yes | Our implementation is based on Ubuntu 20.04, Cuda 12.1, Pytorch 1.12.0, and Pytorch Geometric 2.1.0.post1. |
| Experiment Setup | Yes | Hyper-parameters We Follow the GNNSAFE(Wu et al., 2023) to set the hyper-parameters min, mout, α, η and K. Additionally, we consider grid-search for λ1 {0.0001, 0.001, 0.01, 0.1, 1} and λ2 {0.01, 0.1, 1, 10, 100}, and we use λ1 = 0.001 and λ2 = 1 as the default setting. Training Details In each run, we train the model with 200 epochs as a fixed budget and report the testing performance produced by the epoch where the model yields the lowest classification loss on validation data. |