Evaluating Post-hoc Explanations for Graph Neural Networks via Robustness Analysis
Authors: Junfeng Fang, Wei Liu, Yuan Gao, Zemin Liu, An Zhang, Xiang Wang, Xiangnan He
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive empirical studies validate the effectiveness of our OAR and Sim OAR. |
| Researcher Affiliation | Academia | 1University of Science and Technology of China, 2National University of Singapore |
| Pseudocode | Yes | Algorithm 1 presents the pseudocode of the evaluation process of our proposed method OAR. The pseudocode of Sim OAR can be obtained by removing the line 1, 6, and 8 and simply modifying line 9 into s 1 Nadv P i y(i) . For clarity, we put the step of feeding adversarial graph G (i) into the target GNN and the VGAE under the for-loop. However, when implementing in real code, we can batch all those Nadv adversarial graphs and feed them at one time, after the sampling process is finished, to expedite computation. Meanwhile, Algorithm 2 presents the sampling process of fake explanatory subgraphs for general evaluation. |
| Open Source Code | Yes | Code is available at https://github.com/Mango Killer/Sim OAR_OAR. |
| Open Datasets | Yes | We utilize four benchmark datasets: BA3 [39], TR3 [17], Mutagenicity [40, 41], and MNIST-sp [38], which are publicly accessible and vary in terms of domain and size. |
| Dataset Splits | Yes | Before training, we randomly split BA3, TR3, and Mutagenicity into train and test sets with ratios of 90% and 10%, respectively, while adopting the split provided by Py G for MNIST-sp. During training, we reserve data of the same size as the test set from the train set as the validation set and save the model which reaches the highest classification accuracy on the validation set for later use. |
| Hardware Specification | Yes | All experiments are conducted on a Linux machine with 8 NVIDIA Ge Force RTX 3090 (24 GB) GPUs. CUDA version is 11.6 and Driver Version is 510.39.01. |
| Software Dependencies | Yes | All codes are written under Python 3.9.13 with Py Torch 1.13.0 and Py Torch Geometric (Py G)[65] 2.2.0. We adopt the Adam optimizer throughout all experiments. |
| Experiment Setup | Yes | The target GNNs for BA3, TR3, and Mutagenicity have the same structure, which is a two-layered GIN followed by a two-layered MLP with 32 hidden channels. They are trained with max epochs equal to 20, 200, and 200 respectively, batch size equal to 128, and learning rate equal to 0.001. The target GNN for MNIST-sp is adapted from an example code1 provided by Py G, trained with the number of epochs equal to 20, batch size equal to 64, and initial learning rate equal to 0.01. |