Rethinking Graph Transformers with Spectral Attention
Authors: Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, Prudencio Tossou
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | When tested empirically on a set of 4 standard datasets, our model performs on par or better than state-of-the-art GNNs, and outperforms any attention-based model by a wide margin, becoming the first fully-connected architecture to perform well on graph benchmarks. |
| Researcher Affiliation | Collaboration | Devin Kreuzer Mc Gill University, Mila Montreal, Canada devin.kreuzer@mail.mcgill.ca Dominique Beaini * Valence Discovery Montreal, Canada dominique@valencediscovery.com William L. Hamilton Mc Gill University, Mila Montreal, Canada wlh@cs.mcgill.ca Vincent Létourneau University of Ottawa Ottawa, Canada vletour2@uottawa.ca Prudencio Tossou Valence Discovery Montreal, Canada prudencio@valencediscovery.com |
| Pseudocode | No | The paper contains diagrams (Figure 1 and Figure 4) and mathematical equations, but no structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | An overview of the entire method is presented in Figure 1, with a link to the code here: https://github.com/Devin Kreuzer/SAN. |
| Open Datasets | Yes | The model is implemented in Py Torch [31] and DGL [38] and tested on established benchmarks from [15] and [21] provided under MIT license. Specifically, we applied our method on ZINC, PATTERN, CLUSTER, Mol HIV and Mol PCBA, while following their respective training protocols with minor changes, as detailed in the appendix B.1. |
| Dataset Splits | Yes | Specifically, we applied our method on ZINC, PATTERN, CLUSTER, Mol HIV and Mol PCBA, while following their respective training protocols with minor changes, as detailed in the appendix B.1. Each box plot consists of 4 runs, with different seeds (except Mol HIV). |
| Hardware Specification | Yes | The computation time and hardware is provided in appendix B.4. |
| Software Dependencies | No | The paper states 'The model is implemented in Py Torch [31] and DGL [38]' but does not provide specific version numbers for these software components or any other libraries. |
| Experiment Setup | Yes | Specifically, we applied our method on ZINC, PATTERN, CLUSTER, Mol HIV and Mol PCBA, while following their respective training protocols with minor changes, as detailed in the appendix B.1. For a given dataset, all models use the same hyperparameters, but the hidden dimensions are adjusted to have 500k learnable parameters. |