Spatio-Spectral Graph Neural Networks
Authors: Simon Markus Geisler, Arthur Kosmala, Daniel Herbst, Stephan Günnemann
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | S2GNNs outperform spatial MPGNNs, graph transformers, and graph rewirings, e.g., on the peptide long-range benchmark tasks, and are competitive with state-of-the-art sequence modeling. On a 40 GB GPU, S2GNNs scale to millions of nodes. |
| Researcher Affiliation | Academia | Simon Geisler , Arthur Kosmala , Daniel Herbst, and Stephan Günnemann Department of Computer Science & Munich Data Science Institute Technical University of Munich {s.geisler, a.kosmala, d.herbst, s.guennemann}@tum.de |
| Pseudocode | Yes | In Algo. 1, we provide pseudo code for S2GNNs (Eq. 1). In Algo. 2, we provide pseudo code for spectral filter. |
| Open Source Code | Yes | We provide code at https://www.cs.cit.tum.de/daml/s2gnn. |
| Open Datasets | Yes | The provided code will download all datasets along with the experiment execution, except for TPUGraphs, where one should follow the official instructions. We use the fixed public splits for all experiments and proceed accordingly for our datasets (see M.6 and M.7). |
| Dataset Splits | Yes | We use the fixed public splits for all experiments and proceed accordingly for our datasets (see M.6 and M.7). We sample 25,000/500 random graphs for train/validation. We sample 50,000 training and 2,500 val/test graphs each. |
| Hardware Specification | Yes | For the experiments of 4.1 require <11 GB (e.g. Nvidia GTX 1080Ti); for the experiments in 4.2 & 4.3 we use a 40 GB A100. The cost of partial EVD for each dataset (excluding TPUGraphs and distance regression) is between 1 to 30 minutes on CPUs. |
| Software Dependencies | No | The paper mentions software like "Py Torch geometric" and "scipy" and "Adam W optimizer" but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | We usually optimize weights with Adam W (Loshchilov & Hutter, 2019) and cosine annealing scheduler (Loshchilov & Hutter, 2017). We use early stopping based on the validation loss/score. Peptides: We train for 250 epochs with a batch size of 200. Throughout all evaluations, we maintain a consistent hyperparameter configuration: Specifically, we use an inner dimension of 128, GELU (Hendrycks & Gimpel, 2016) as an activation function, no dropout, and residual connections for all spatial and spectral layers. We train for 50 epochs with a batch size of 50, using the Adam W optimizer (Loshchilov & Hutter, 2019) with a base learning rate of 0.003, a weight decay of 0.0001, a cosine scheduler and 5 warmup epochs. |