Enhancing Graph Transformers with Hierarchical Distance Structural Encoding
Authors: Yuankai Luo, Hongkang Li, Lei Shi, Xiao-Ming Wu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we demonstrate that graph transformers with HDSE excel in graph classification, regression on 7 graph-level datasets, and node classification on 11 large-scale graphs. |
| Researcher Affiliation | Academia | Yuankai Luo1,3 Hongkang Li2 Lei Shi1 Xiao-Ming Wu3 1Beihang University 2Rensselaer Polytechnic Institute 3The Hong Kong Polytechnic University luoyk@buaa.edu.cn xiao-ming.wu@polyu.edu.hk |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | Our implementation is available at https://github.com/LUOyk1999/HDSE. |
| Open Datasets | Yes | Datasets. We consider various graph learning tasks from popular benchmarks as detailed below and in Appendix B. Graph-level Tasks. For graph classification and regression, we evaluate our method on five datasets from Benchmarking GNNs [20]: ZINC, MNIST, CIFAR10, PATTERN, and CLUSTER. We also test on two peptide graph benchmarks from the Long-Range Graph Benchmark [23]: Peptides-func and Peptides-struct... Node Classification on Large-scale Graphs. We consider node classification over the citation graphs Cora, Cite Seer and Pub Med [44], the Actor co-occurrence graph [14], and the Squirrel and Chameleon page-page networks from [71], all of which have 1K-20K nodes. Further, we extend our evaluation to larger datasets from the Open Graph Benchmark (OGB) [35]: ogbn-arxiv, arxiv-year, ogbn-papers100M, ogbn-proteins and ogbn-products... |
| Dataset Splits | Yes | For each dataset, we follow the standard train/validation/test splits and evaluation metrics in [68]. For more comprehensive details, readers are encouraged to refer to [68]. Cora, Citeseer, Pubmed, Actor, Squirrel, Chameleon, ogbn-proteins, ogbn-arxiv, ogbn-products and ogbn-papers100M. For each dataset, we use the same train/validation/test splits and evaluation metrics as [82]. Arxiv-year is a citation network among all computer science arxiv papers, as described by [55]. We use the public splits shared by [55], with a train/validation/test split ratio of 50%/25%/25%. |
| Hardware Specification | Yes | Our implementation is based on Py G [25] and DGL [76]. The experiments are conducted on a single workstation with 4 RTX 3090 GPUs and a quad-core CPU. |
| Software Dependencies | No | The paper mentions software like "Py G [25] and DGL [76]" but does not specify their version numbers, which is necessary for a reproducible description of ancillary software. |
| Experiment Setup | Yes | For fair comparisons, we use the same hyperparameters (including the number of layers, hidden dimension etc.), PE and readout as the baseline transformers. For GOAT + HDSE, Tables 9, 10 and 11 showcase the corresponding hyperparameters and coarsening algorithms. We fix the maximum distance length L = 30 and vary the maximum hierarchy level K in {0, 1, 2} in all experiments. |