Structure-Aware Transformer for Graph Representation Learning
Authors: Dexiong Chen, Leslie O’Bray, Karsten Borgwardt
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, our method achieves state-of-the-art performance on five graph prediction benchmarks. Our structure-aware framework can leverage any existing GNN to extract the subgraph representation, and we show that it systematically improves performance relative to the base GNN model, successfully combining the advantages of GNNs and Transformers. Our code is available at https: //github.com/Borgwardt Lab/SAT. |
| Researcher Affiliation | Academia | 1Department of Biosystems Science and Engineering, ETH Z urich, Switzerland 2SIB Swiss Institute of Bioinformatics, Switzerland. |
| Pseudocode | No | The paper describes methods using prose and mathematical equations but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https: //github.com/Borgwardt Lab/SAT. |
| Open Datasets | Yes | We assess the performance of our method with five medium to large benchmark datasets for node and graph property prediction, including ZINC (Dwivedi et al., 2020), CLUSTER (Dwivedi et al., 2020), PATTERN (Dwivedi et al., 2020), OGBG-PPA (Hu et al., 2020a) and OGBGCODE2 (Hu et al., 2020a). |
| Dataset Splits | Yes | For each dataset, we follow their respective training protocols and use the standard train/validation/test splits and evaluation metrics. In all experiments, we use the validation set to select the dropout rate and the size of the subtree or subgraph k {1, 2, 3, 4}. |
| Hardware Specification | Yes | All experiments were performed on a shared GPU cluster equipped with GTX1080, GTX1080TI, GTX2080TI and TITAN RTX. |
| Software Dependencies | No | The paper mentions using the 'Adam W optimizer (Loshchilov & Hutter, 2018)' but does not provide specific version numbers for software dependencies such as programming languages, deep learning frameworks (e.g., PyTorch, TensorFlow), or other relevant libraries. |
| Experiment Setup | Yes | The hyperparameters for training SAT models on different datasets are summarized in Table 4, where only the dropout rate and the size of the subgraph k are tuned (k {1, 2, 3, 4}). All other hyperparameters are fixed for simplicity, including setting the readout method to mean pooling. |