Energy-based Out-of-Distribution Detection for Graph Neural Networks
Authors: Qitian Wu, Yiting Chen, Chenxiao Yang, Junchi Yan
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | For comprehensive evaluation, we introduce new benchmark settings that evaluate the model for detecting OOD data from both synthetic and real distribution shifts (crossdomain graph shifts and temporal graph shifts). The results show that GNNSAFE achieves up to 17.0% AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area. |
| Researcher Affiliation | Academia | Qitian Wu, Yiting Chen, Chenxiao Yang, Junchi Yan Department of CSE & Mo E Lab of Artificial Intelligence, Shanghai Jiao Tong University {echo740, sjtucyt, chr26195, yanjunchi}@sjtu.edu.cn |
| Pseudocode | No | The paper describes methods through mathematical equations and textual explanations, but it does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The codes are available at https://github.com/qitianwu/Graph OOD-GNNSafe. |
| Open Datasets | Yes | The datasets used in our experiment are all public available as common benchmarks for evaluating graph learning models. For ogbn-Arxiv, we use the preprocessed dataset and data loader provided by the OGB package2. For other datasets, we use the data loader provided by the Pytorch Geometric package3. |
| Dataset Splits | Yes | For in-distribution data in each dataset, only partial nodes labels are exposed for supervised training and another subset of nodes is used for validation. Other nodes are used as in-distribution testing data. We provide detailed dataset and splitting information in Appendix B.1. ... For Amazon and Coauthor... we follow the common practice (Kipf & Welling, 2017) and use random splits with 1:1:8 for training/validation/testing. |
| Hardware Specification | Yes | Most of the experiments are running with a NVIDIA 2080Ti with 11GB memory, except that for cases where the model requires larger GPU memory we use a NVIDIA 3090 with 24GB memory for experiments. |
| Software Dependencies | Yes | Our implementation is based on Ubuntu 16.04.6, Cuda 10.2, Pytorch 1.9.0 and Pytorch Geometric 2.0.3. |
| Experiment Setup | Yes | We basically set the propagation layer number K as 2 and weight α as 0.5. For fair comparison, the GCN model with layer depth 2 and hidden size 64 is used as the backbone encoder for all the model. ... For GNNSAFE, we only consider grid-search for the learning rate within {0.1, 0.01, 0.001} using validation set. For GNNSAFE++, we use the validation set for additionally search for margin hyper-parameters tin {-9, -7, -5}, tout {-1, -2, -3, -4} and the regularization weight λ {0.01, 0.1, 1.0}. |