Empowering Graph Invariance Learning with Deep Spurious Infomax
Authors: Tianjun Yao, Yongqiang Chen, Zhenhao Chen, Kai Hu, Zhiqiang Shen, Kun Zhang
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on 7 synthetic datasets and 8 real-world datasets with various types of distribution shifts. The results demonstrate the superiority of our method compared to state-of-the-art approaches. |
| Researcher Affiliation | Academia | 1Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE 2The Chinese University of Hong Kong, Hong Kong, China 3Carnegie Mellon University, Pittsburgh, USA. |
| Pseudocode | Yes | Appendix G. Algorithmic Pseudocode Algorithm 1 The EQu AD Framework |
| Open Source Code | Yes | Our code is available at https://github.com/tianyao-aka/EQu AD. |
| Open Datasets | Yes | To comprehensively evaluate our proposed method under two data generating assumptions, namely FIIF and PIIF, we utilize the SPMotif datasets (Wu et al., 2022b) and Two-piece graph datasets (Chen et al., 2023) to verify its effectiveness. Additionally, for real-world datasets, we employ the Drug OOD datasets (Ji et al., 2022), which focus on the challenging task of AI-aided drug affinity prediction. ... Moreover, we also consider two molecule datasets, Mol BACE and Mol BBBP, from Open Graph Benchmark (Hu et al., 2021)... |
| Dataset Splits | Yes | This section delves into the quality of the latent representation derived from the encoder h( ) in step 3. We explore the distribution discrepancy between latent embeddings from the training set and those from the validation set. |
| Hardware Specification | Yes | We ran all our experiments on Linux Servers with Ge Force RTX 4090 with CUDA 11.8. |
| Software Dependencies | Yes | We implement our method and all baseline methods using Py Torch(Paszke et al., 2019) and Py Torch Geometric(Fey & Lenssen, 2019). ... Py GCL(Zhu et al., 2021) package for the implementation. We utilize linear SVC and Calibrated Classifier CV in scikit-learn(Pedregosa et al., 2011) for the implementation of linear SVM and probability calibration classifier respectively. |
| Experiment Setup | Yes | For encoding stage...we set teleport probability α = 0.2, and diffusion time t = 5. To obtain a collection of latent representation H, we set pre-defined training epochs P = {50, 100, 150}, number of layers in GNN encoder L = {2, 3, 5}, hidden dimensions H = {32, 64}...the regularization parameters C of linear svms are searched over {10, 1000}. ... For the decorrelation step, we utilize Eqn. 8 and search over the following hyperparameters: γ : {0.7, 0.5, 0.3, 0.1, 0.05}, τ : {1.0, 0.5, 0.25, 0.1, 0.01}, λ : {1e-3, 1e-2, 1e-1}. ...the number of layers L are searched over {3, 5}...hidden dimension is uniformly set to 32 across all methods, while for the Drug OOD and OGBG datasets, it is set to 128. ...the experiments are ran for three times, with random seeds {1, 2, 3}. The batch size is 32 for all experiments, with a learning rate of 1e-3 is applied across all experiments. We adopt Adam optimizer (Kingma & Ba, 2017) for model training. For EQu AD, the experiments for SPMotif and Drug OOD datasets are ran for 50 epochs, and for OGBG datasets, the experiments are ran for 100 epochs. |