Expressivity-Preserving GNN Simulation
Authors: Fabian Jogl, Maximilian Thiessen, Thomas Gärtner
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical evaluation shows competitive predictive performance of message passing on transformed graphs for various molecular benchmark datasets, in several cases surpassing the original GNNs. |
| Researcher Affiliation | Academia | Fabian Jogl Machine Learning Research Unit Center for Artificial Intelligence and Machine Learning TU Wien, Vienna, Austria fabian.jogl@tuwien.ac.at Maximilian Thiessen Machine Learning Research Unit TU Wien, Vienna, Austria maximilian.thiessen@tuwien.ac.at Thomas G artner Machine Learning Research Unit TU Wien, Vienna, Austria thomas.gaertner@tuwien.ac.at |
| Pseudocode | Yes | Algorithm 1 Helper algorithm for atoms; Algorithm 2 Helper algorithm for (S2); Algorithm 3 Helper algorithm for AMP; Algorithm 4 Augmented message encoding (AME); Algorithm 5 Rgraph for generalized augmented message encoding; Algorithm 6 Generalized augmented message encoding (g AME) |
| Open Source Code | Yes | The code for our experiments can be found at https://github.com/ocatias/GNN-Simulation. |
| Open Datasets | Yes | For real world prediction tasks, we use all real world datasets with less than 105 graphs that provide a train, validation, and test split used in Bodnar et al. [2021a], Bevilacqua et al. [2021]: ZINC [G omez-Bombarelli et al., 2018, Sterling and Irwin, 2015], ogbg-molhiv and ogbg-moltox21 [Hu et al., 2020]. Additionally, we add seven small molecule datasests from OGB [Hu et al., 2020]. In total, we evaluate on 10 real-life datasets. |
| Dataset Splits | Yes | For real world prediction tasks, we use all real world datasets with less than 105 graphs that provide a train, validation, and test split used in Bodnar et al. [2021a], Bevilacqua et al. [2021]...For real-life datasets we combine all baseline models (GCN, GIN) with all graph transformations (DS, DSS, CWN) and tune hyperparameters individually (including CWN, DS and DSS; see Appendix H). We also measure the preprocessing and training speeds for different models (details and speed results are in Appendix H.3). |
| Hardware Specification | Yes | CWN training and all speed evaluations are performed on an an NVIDIA Ge Force GTX 1080 GPU. All other experiments are performed on NVIDIA Ge Force RTX 3080 GPUs. |
| Software Dependencies | No | All models are implemented in Pytorch Geometric [Fey and Lenssen, 2019]. We use Wand B [Biewald, 2020] to track our experiments. Unfortunately, CWN requires an old version of Py Torch [Paszke et al., 2019] meaning that we have to train it on older GPUs. While specific software is mentioned, version numbers for all key components are not consistently provided (e.g., PyTorch Geometric, WandB, PyTorch versions are implied or only mentioned for older versions without full specification for all experiments). |
| Experiment Setup | Yes | Table 3 contains all hyperparameter grids for the real life tasks. During preliminary experiments we found that it is crucial to tune the pooling operation and number of layers to achieve strong results. For all OGB datasets, we train with a batch size of 32 for 100 epochs with a fixed learning rate to allow for fast training of many models. For ZINC we train as described by Bevilacqua et al. [2021], Bodnar et al. [2021a] and Dwivedi et al. [2023] i.e., for up to 500 epochs with a batch size of 128 with an initial learning rate of 10 3 that reduces by a factor of 0.5 every time the validation result has not improved for 20 epochs. If the learning rate dips below 10 5 the training stops. |