Scalable and Effective Implicit Graph Neural Networks on Large Graphs
Authors: Juncheng Liu, Bryan Hooi, Kenji Kawaguchi, Yiwei Wang, Chaosheng Dong, Xiaokui Xiao
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comprehensive experiments on various large graphs demonstrate that SEIGNN outperforms baselines and achieves higher accuracy with less training time compared with existing implicit GNNs. In this section, we demonstrate the effectiveness and efficiency of SEIGNN compared with both implicit GNNs and representative traditional GNNs on large graph datasets for the node classification task. Specifically, we conduct experiments on 6 commonly used datasets for node classification (i.e., Flickr, Yelp, Reddit, PPI, ogbn-arxiv, and ogbn-products). |
| Researcher Affiliation | Collaboration | Juncheng Liu Bryan Hooi Kenji Kawaguchi Yiwei Wang Chaosheng Dong Xiaokui Xiao National University of Singapore University of California, Los Angeles Amazon.com Inc. |
| Pseudocode | Yes | Algorithm 1: The stochastic solver s procedure for the equilibrium. Input: The subgraph G, the normalized adjacency matrix S, and the node features X. Output: The approximated equilibrium ˆZ . |
| Open Source Code | No | The paper does not contain any statements about releasing source code for the described methodology, nor does it provide a link to a code repository. |
| Open Datasets | Yes | we conduct experiments on 6 commonly used datasets for node classification (i.e., Flickr, Yelp, Reddit, PPI, ogbn-arxiv, and ogbn-products). We download ogbn-arxiv dataset from the OGB website 5. We download ogbn-products dataset from the OGB website 5. |
| Dataset Splits | Yes | The dataset splits used in our paper are the same as in Hamilton et al. (2017), i.e., 20 graphs for training, 2 graphs for validation, and 2 graphs for testing. Table 8: Dataset statistics. Train/Val/Test for Flickr: 0.50/0.25/0.25, Reddit: 0.66/0.10/0.24, Yelp: 0.75/0.10/0.15, PPI: 0.79/0.11/0.10, ogbn-arxiv: 0.54/0.18/0.29, ogbn-products: 0.10/0.02/0.88. |
| Hardware Specification | Yes | We mainly run the experiments on an RTX-A5000 GPU with 24GB GPU memory. |
| Software Dependencies | No | The paper mentions the use of the Adam optimizer: "The Adam optimizer (Kingma and Ba, 2015) is used for optimization." However, it does not provide specific version numbers for any programming languages, libraries, or other software dependencies used in the experiments (e.g., Python version, PyTorch version). |
| Experiment Setup | Yes | For SEIGNN, we use the same structure with a few implicit graph layers and the same number of linear layers as in MGNNI (Liu et al., 2022) and USP (Li et al., 2023). We select the number of implicit graph layers from {2, 3, 4}. We also conduct a hyperparameter search on learning rate {0.01, 0.005, 0.001} and dropout rate {0.0, 0.2, 0.5}. The number of deterministic steps t in our stochastic solver is chosen from {3, 5} and the continuation probability α is set to 0.5. The hyperparameter γ used in an implicit graph layer is set to 0.8. The Adam optimizer (Kingma and Ba, 2015) is used for optimization. The number of partitions for adding coarse nodes in our mini-batch training method is selected from {50, 100, 200}. The number of target nodes in a mini-batch is configured as follows: 8192 for Flickr and PPI, 10240 for ogbn-arxiv, Yelp, and Reddit, and 16384 for ogbn-products. |