Beltrami Flow and Neural Diffusion on Graphs
Authors: Benjamin Chamberlain, James Rowbottom, Davide Eynard, Francesco Di Giovanni, Xiaowen Dong, Michael Bronstein
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experimental results In this section, we compare the proposed Beltrami framework to popular GNN architectures on standard node classification benchmarks and provide a detailed study of the choice of the positional encoding space. |
| Researcher Affiliation | Collaboration | Benjamin P. Chamberlain Twitter Inc. James Rowbottom Twitter Inc. Davide Eynard Twitter Inc. Francesco Di Giovanni Twitter Inc. Xiaowen Dong University of Oxford Michael M. Bronstein Twitter Inc. and Imperial College London |
| Pseudocode | No | The paper does not contain any explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available at https://github.com/twitter-research/graph-neural-pde. |
| Open Datasets | Yes | In our experiments, we use the following datasets: Cora [56], Citeseer [77], Pubmed [58], Coauthor CS [79], Amazon, Computer, and Photo [55], and OGB-arxiv [35]. |
| Dataset Splits | Yes | Hyperparameters with the highest validation accuracy were chosen and results are reported on a test set that is used only once. Where available, results from [79] are reported. Hyperparameter search used Ray Tune [50] with a thousand random trials using an asynchronous hyperband scheduler with a grace period and half life of ten epochs. Also 'Datasets ... Since many works using the first three datasets rely on the Planetoid splits [96], we included them Table 2, together with a more robust evaluation on 100 random splits with 20 random initialisations [79].' |
| Hardware Specification | Yes | Experiments ran on AWS p2.8xlarge machines, each with 8 Tesla V100-SXM2 GPUs. |
| Software Dependencies | No | The paper mentions software like 'adaptive explicit Dormand-Prince scheme', 'Runge-Kutta method', 'Ray Tune [50]', 'Py Torch Geometric s Node2Vec implementation', and 'HGCN [17] implementation'. However, it does not provide specific version numbers for these software components, which is required for a reproducible description. |
| Experiment Setup | Yes | Hyperparameters with the highest validation accuracy were chosen and results are reported on a test set that is used only once. Hyperparameter search used Ray Tune [50] with a thousand random trials using an asynchronous hyperband scheduler with a grace period and half life of ten epochs. For all datasets excepting ogb-arxiv, adaptive explicit Dormand-Prince scheme was used as the numerical solver; for ogb-arxiv, we used the Runge-Kutta method. For the two smallest datasets (Cora and Citeseer) we performed direct backpropagation through each step of the numerical integrator. For the larger datasets, to reduce memory complexity, we use Pontryagin s maximum principle to propagate gradients backwards in time [69]. For the larger datasets, kinetic energy and Jacobian regularisation [28, 37] was employed. We use constant initialisation for the attention weights, WK, WQ, so training starts from a well-conditioned system that induces small regularisation penalty terms [28]. |