PairNorm: Tackling Oversmoothing in GNNs
Authors: Lingxiao Zhao, Leman Akoglu
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on real-world graphs demonstrate that PAIRNORM makes deeper GCN, GAT, and SGC models more robust against oversmoothing, and significantly boosts performance for a new problem setting that benefits from deeper GNNs. |
| Researcher Affiliation | Academia | Lingxiao Zhao Carnegie Mellon University Pittsburgh, PA 15213, USA {lingxia1}@andrew.cmu.edu Leman Akoglu Carnegie Mellon University Pittsburgh, PA 15213, USA {lakoglu}@andrew.cmu.edu |
| Pseudocode | No | The paper describes the PAIRNORM procedure using mathematical equations (Eq. 10 and 11) and an illustration (Figure 2), but does not provide a formally labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | Yes | Code is available at https://github.com/Lingxiao Shawn/Pair Norm. |
| Open Datasets | Yes | Datasets. We use 4 well-known benchmark datasets in GNN domain: Cora, Citeseer, Pubmed (Sen et al., 2008), and Coauthor CS (Shchur et al., 2018). Their statistics are reported in Appx. A.2. |
| Dataset Splits | Yes | For Cora, Citeseer and Pubmed, we use the same dataset splits as Kipf & Welling (2017), where all nodes outside train and validation are used as test set. For Coauthor CS, we randomly split all nodes into train/val/test as 3%/10%/87%, and keep the same split for all experiments. |
| Hardware Specification | Yes | We mainly use a single GTX-1080ti GPU, with some SGC experiments ran on an Intel i7-8700k CPU. |
| Software Dependencies | No | The paper mentions using specific GNN models and Tensorboard, but does not provide specific version numbers for any software dependencies or libraries (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | Hyperparameters. We choose the hyperparameter s of PAIRNORM from {0.1, 1, 10, 50, 100} over validation set for SGC, while keeping it fixed at s = 1 for both GCN and GAT due to resource limitations. We set the #hidden units of GCN and GAT (#attention heads is set to 1) to 32 and 64 respectively for all datasets. Dropout with rate 0.6 and L2 regularization with penalty 5 10 4 are applied to GCN and GAT. Configurations. For PAIRNORM-enhanced models, we apply PAIRNORM after each graph convolution layer (i.e., after activation if any) in the base model... We run each experiment within 1000 epochs 5 times and report the average performance. |