DiGRAF: Diffeomorphic Graph-Adaptive Activation Function
Authors: Krishna Sri Ipsit Mantri, Xinzhi Wang, Carola-Bibiane Schönlieb, Bruno Ribeiro, Beatrice Bevilacqua, Moshe Eliasof
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct an extensive set of experiments across diverse datasets and tasks, demonstrating a consistent and superior performance of DIGRAF compared to traditional and graph-specific activation functions, highlighting its effectiveness as an activation function for GNNs. |
| Researcher Affiliation | Academia | Krishna Sri Ipsit Mantri Purdue University mantrik@purdue.edu Xinzhi (Aurora) Wang Purdue University wang6171@purdue.edu Carola-Bibiane Schönlieb University of Cambridge cbs31@cam.ac.uk Bruno Ribeiro Purdue University ribeirob@purdue.edu Beatrice Bevilacqua Purdue University bbevilac@purdue.edu Moshe Eliasof University of Cambridge me532@cam.ac.uk |
| Pseudocode | No | No explicit pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | Yes | Our code is available at https://github.com/ ipsitmantri/Di GRAF. |
| Open Datasets | Yes | We conduct an extensive set of experiments on a diverse set of datasets across various tasks, including node classification, graph classification, and regression. Our evaluation compares the performance of DIGRAF with three types of baselines: traditional activation functions, activation functions with trainable parameters, and graph activation functions. Our code is available at https://github.com/ ipsitmantri/Di GRAF. The paper references specific public datasets like OGB, ZINC-12K, and TUDatasets, all of which are widely known and publicly accessible in the machine learning community. |
| Dataset Splits | Yes | For each dataset, we train a 2-layer GCN [43] as the backbone architecture, and integrate each of the activation functions into this model. Following Zhang et al. [92], we randomly choose 20 nodes from each class for training and select 1000 nodes for testing. For each activation function, we run the experiment 10 times with random partitions. [...] For hyperparameter tuning and model selection, we utilized the Weights and Biases (wandb) library [6]. |
| Hardware Specification | Yes | All experiments were conducted on NVIDIA RTX A5000, NVIDIA Ge Force RTX 4090, NVIDIA Ge Force RTX 4070 Ti Super, NVIDIA Ge Force GTX 1080 Ti, NVIDIA TITAN RTX and NVIDIA TITAN V GPUs. |
| Software Dependencies | No | The paper mentions software like PyTorch, PyTorch Geometric, Weights and Biases (wandb), and the difw package, but does not provide specific version numbers for these software components. It only provides citations to the papers introducing them. |
| Experiment Setup | Yes | Hyperparameters. The hyperparameters include the number of layers L and embedding dimension C of GNN(l) LAYER, learning rates and weight decay factors for both GNN(l) LAYER and GNNACT, dropout rate p, tessellation size NP, and regularization coefficient λ. We additionally include the number of layers LACT and embedding dimension CACT of GNNACT. We employed a combination of grid search and Bayesian optimization. All hyperparameters were chosen according to the best validation metric. For the baselines, we include only the applicable hyperparameters in our search space. Tables 12, 13, and 15 provide specific hyperparameter configurations and search ranges. |