Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Generalizable Insights for Graph Transformers in Theory and Practice
Authors: Timo Stoll, Luis Müller, Christopher Morris
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experiments, we identify design choices that consistently perform well across various applications, tasks, and model scales, demonstrating strong performance in a few-shot transfer setting without fine-tuning. Our evaluation covers over eight million graphs with roughly 270M tokens across diverse domains, including image-based object detection, molecular property prediction, code summarization, and out-of-distribution algorithmic reasoning. |
| Researcher Affiliation | Academia | Timo Stoll Luis Müller Christopher Morris Department of Computer Science RWTH Aachen University Aachen, Germany EMAIL |
| Pseudocode | No | The paper describes the GDT architecture using mathematical definitions and textual descriptions (e.g., in Section 2.2 and Appendix B.1) but does not include explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | We provide our code base in the supplementary material which is sufficient to download or generate the data required to reproduce the experiments. |
| Open Datasets | Yes | PCQM4MV2 is available at https://ogb.stanford.edu/docs/lsc/pcqm4mv2/ under a CC BY 4.0 license. OGB-Code2 is available at https://ogb.stanford.edu/docs/graphprop/ #ogbg-code2 under a MIT license. The COCO-SP and PASCAL-VOC-SP datasets as part of the LRGB benchmark [Dwivedi et al., 2022] are available at https://github.com/vijaydwivedi75/ lrgb under a CC BY 4.0 license. BREC is available at https://github.com/Graph PKU/BREC under a MIT license. |
| Dataset Splits | No | Across the experiments, we select the hyperparameters based on the best validation score and then evaluate on the test set. While training, validation, and test sets are used, the paper does not explicitly state the dataset split percentages, absolute sample counts for each split, or reference predefined standard splits with specific methodologies or citations for the datasets used in their experiments. |
| Hardware Specification | Yes | We run the experiments on a single node consisting of one L40 GPU with 40GB VRAM, 12 CPU cores, and 120GB RAM for all runtime and memory computations. In case of COCO and CODE with RRWP as a PE, we used 2 l40 GPUs. |
| Software Dependencies | No | We base our implementation on the torch.nn.Transformer Encoder Layer proposed in Py Torch [Paszke et al., 2019]. The paper mentions PyTorch and Flash Attention, but does not provide specific version numbers for these or other key software components like Python or CUDA. |
| Experiment Setup | Yes | Table 3: Hyperparameters for our 16M models. (lists specific values for Learning rate, Batch size, Optimizer, Num. layers, Hidden dim., Num. heads, Activation, Dropout, etc., across all tasks). Table 4: Hyperparameters for our 90M and 160M models. (provides similar details for scaled models). |