Specformer: Spectral Graph Neural Networks Meet Transformers
Authors: Deyu Bo, Chuan Shi, Lele Wang, Renjie Liao
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On synthetic datasets, we show that our Specformer can better recover ground-truth spectral filters than other spectral GNNs. Extensive experiments of both node-level and graph-level tasks on real-world graph datasets show that our Specformer outperforms state-ofthe-art GNNs and learns meaningful spectrum patterns. |
| Researcher Affiliation | Academia | Deyu Bo1, Chuan Shi1 , Lele Wang2, Renjie Liao2 Beijing University of Posts and Telecommunications1, University of British Columbia2 {bodeyu, shichuan}@bupt.edu.cn, {lelewang, rjliao}@ece.ubc.ca |
| Pseudocode | No | The paper describes the Specformer architecture and its components (e.g., eigenvalue encoding, decoding, graph convolution) in detail, but it does not provide an explicitly labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | Yes | Code and data are available at https://github.com/bdy9527/Specformer. |
| Open Datasets | Yes | Mol HIV and Mol PCBA are taken from the Open Graph Benchmark (OGB) datasets (Hu et al., 2020). PCQM4Mv2 is a large-scale graph regression dataset (Hu et al., 2021). |
| Dataset Splits | Yes | For all datasets, we use the full-supervised split, i.e., 60% for training, 20% for validation, and 20% for testing, as suggested in (He et al., 2021). |
| Hardware Specification | Yes | CPU information: Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz Ge Force RTX 3090 (24GB) |
| Software Dependencies | Yes | Operating system: Linux version 3.10.0-693.el7.x86 64 |
| Experiment Setup | Yes | Hyperparameters. The hyperparameters of specformer can be seen in Tables 7 and 8. For the node classification task, we use the Adam (Kingma & Ba, 2015) optimizer... For graph-level tasks, we use the Adam W (Loshchilov & Hutter, 2019) optimizer, with the default parameters of ϵ =1e-8 and (β1, β2) = (0.99, 0.999), as suggested by Ying et al. (2022); Ramp asek et al. (2022). Besides, we also use a learning rate scheduler for graph-level tasks, which is first a linear warm-up stage followed by a cosine decay. |