GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
Authors: Tianle Cai, Shengjie Luo, Keyulu Xu, Di He, Tie-Yan Liu, Liwei Wang
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate the effectiveness of Graph Norm on eight popular graph classification benchmarks. Empirical results confirm that Graph Norm consistently improves the speed of converge and stability of training for GNNs compared to those with Batch Norm, Instance Norm, Layer Norm, and those without normalization. |
| Researcher Affiliation | Collaboration | 1Department of Electrical and Computer Engineering, Princeton University 2Zhongguancun Haihua Institute for Frontier Information Technology 3Key Laboratory of Machine Perception, MOE, School of EECS, Peking University 4Center for Data Science, Peking University 5Pazhou Lab 6CSAIL, Massachusetts Institute of Technology (MIT) 7Microsoft Research. |
| Pseudocode | No | The paper includes mathematical formulations and descriptions of methods, but it does not present any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any statements or links indicating the availability of open-source code for the described methodology. |
| Open Datasets | Yes | We use eight popularly used benchmark datasets of different scales in the experiments (Yanardag & Vishwanathan, 2015; Xu et al., 2019), including four medium-scale bioinformatics datasets (MUTAG, PTC, PROTEINS, NCI1), three medium-scale social network datasets (IMDB-BINARY, COLLAB, REDDIT-BINARY), and one large-scale bioinformatics dataset ogbg-molhiv, which is recently released on Open Graph Benchmark (OGB) (Hu et al., 2020). |
| Dataset Splits | No | The paper mentions using benchmark datasets and states 'Details of the experimental settings are presented in Appendix C.' However, within the provided text, specific training, validation, or test split percentages or counts are not explicitly detailed. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper mentions model architectures like GIN and GCN and uses terms like 'MLP' and 'ReLU', implying the use of deep learning frameworks. However, it does not specify any software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We set batch size to 128. For GIN, the number of sub-layers in MLP is set to 2. Normalization is applied to each layer. To aggregate global features on top of the network, we use SUM readout for MUTAG, PTC, PROTEINS and NCI1 datasets, and use MEAN readout for other datasets, as in Xu et al. (2019). Details of the experimental settings are presented in Appendix C. |