Design Space for Graph Neural Networks
Authors: Jiaxuan You, Zhitao Ying, Jure Leskovec
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Here we define and systematically study the architectural design space for GNNs which consists of 315,000 different designs over 32 different predictive tasks. Our key results include: (1) A comprehensive set of guidelines for designing well-performing GNNs; (2) while best GNN designs for different tasks vary significantly, the GNN task space allows for transferring the best designs across different tasks; (3) models discovered using our design space achieve state-of-the-art performance. (Abstract) and We evaluate the proposed GNN design space (Section 4) over the GNN task space (Section 5), using the proposed evaluation techniques (Section 6). For all the experiments in Sections 7.3 and 7.4, we use a consistent setup, where results on three random 80%/20% train/val splits are averaged, and the validation performance in the final epoch is reported. (Section 7.2) |
| Researcher Affiliation | Academia | Jiaxuan You Rex Ying Jure Leskovec Department of Computer Science, Stanford University {jiaxuan, rexy, jure}@cs.stanford.edu |
| Pseudocode | No | The paper does not contain any sections, figures, or blocks explicitly labeled "Pseudocode" or "Algorithm", nor does it present structured code-like steps for any procedure. |
| Open Source Code | Yes | Finally, we release Graph Gym, a powerful platform for exploring different GNN designs and tasks. Graph Gym features modularized GNN implementation, standardized GNN evaluation, and reproducible and scalable experiment management. (Abstract) and 2https://github.com/snap-stanford/graphgym (Footnote 2) |
| Open Datasets | Yes | To properly evaluate the proposed design space and the proposed task similarity metric, we collect a variety of 32 synthetic and real-world GNN tasks/datasets. (Section 5.2) and We include 6 node classification benchmarks from [26, 28], and 6 graph classification tasks from [14]. (Section 5.2) and 1Project website with data and code: http://snap.stanford.edu/gnn-design (Footnote 1). |
| Dataset Splits | Yes | For all the experiments in Sections 7.3 and 7.4, we use a consistent setup, where results on three random 80%/20% train/val splits are averaged, and the validation performance in the final epoch is reported. (Section 7.2) and We use the train/val/test splits provided by the dataset, and report the test accuracy in the final epoch. (Section 7.5) |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware used for running the experiments, such as GPU models, CPU types, or cloud computing instance specifications. |
| Software Dependencies | No | The paper does not list specific software dependencies with their version numbers, such as programming languages, libraries (e.g., PyTorch, TensorFlow), or solvers. |
| Experiment Setup | Yes | Training configurations. Optimization algorithm plays an important role in GNN performance. In GNN literature, training configurations including batch size, learning rate, optimizer type and training epochs often vary a lot. Here we consider the following design dimensions for GNN training: Batch size 16, 32, 64; Learning rate 0.1, 0.01, 0.001; Optimizer SGD, ADAM; Training epochs 100, 200, 400 (Section 4) and Table 1: Condensed GNN design space based on the analysis in Section 7.3... Batch 32 LR 0.01 Optimizer ADAM Epoch 400 (Section 7.4) |