Orthogonal Graph Neural Networks
Authors: Kai Guo, Kaixiong Zhou, Xia Hu, Yu Li, Yi Chang, Xin Wang3996-4004
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through a number of experimental observations, we argue that the main factor degrading the performance is the unstable forward normalization and backward gradient resulted from the improper design of the feature transformation, especially for shallow GNNs where the over-smoothing has not happened. Therefore, we propose a novel orthogonal feature transformation, named Ortho GConv, which could generally augment the existing GNN backbones to stabilize the model training and improve the model s generalization performance. Specifically, we maintain the orthogonality of the feature transformation comprehensively from three perspectives, namely hybrid weight initialization, orthogonal transformation, and orthogonal regularization. By equipping the existing GNNs (e.g. GCN, JKNet, GCNII) with Ortho-GConv, we demonstrate the generality of the orthogonal feature transformation to enable stable training, and show its effectiveness for node and graph classification tasks. |
| Researcher Affiliation | Academia | 1School of Artificial Intelligence, Jilin University, Changchun, China 2Department of Computer Science, Rice University, USA 3College of Computer Science and Technology, Jilin University, Changchun, China 4International Center of Future Science, Jilin University |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. Methods are described through mathematical equations and prose. |
| Open Source Code | Yes | Our code is publicly available 1. [1] https://github.com/Kai Guo20/Ortho-GConv |
| Open Datasets | Yes | For full-supervised node classification task, we use Cora (Sen et al. 2008), Cite Seer (Sen et al. 2008), Pub Med (Sen et al. 2008), and three sub-sets of Web KB (Pei et al. 2020): Cornell, Texas and Wisconsin to evaluate the performance. For graph classification task, we use the protein datasets including D&D (Dobson and Doig 2003) and PROTEINS (Borgwardt et al. 2005). In addition, we conduct experiments on ogbn-arxiv (Hu et al. 2020a) to evaluate the scalability and performance of Ortho-GConv on large graph. |
| Dataset Splits | Yes | For each dataset, we randomly split the nodes of each class into 60%, 20% and 20% for training, validation and testing, respectively. In addition, we conduct experiments on ogbn-arxiv to further evaluate the performance of our proposed Ortho GConv on large graph. For this dataset, we split the nodes of each class into 54%, 18% and 28% for training, validation and testing by following the previous effort (Hu et al. 2020b). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or specific computing environments) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers (e.g., library or solver names with version numbers like Python 3.8, PyTorch 1.9) needed to replicate the experiment. |
| Experiment Setup | Yes | We follow the same experimental settings as Graph-U-Nets for a fair comparison, and fix the parameters T = 4, β = 0.4. The statistics of datasets and parameter settings are provided in Appendix.3 and 4, respectively. |