Graph Contrastive Learning with Augmentations
Authors: Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We then systematically study the impact of various combinations of graph augmentations on multiple datasets, in four different settings: semi-supervised, unsupervised, and transfer learning as well as adversarial attacks. The results show that, even without tuning augmentation extents nor using sophisticated GNN architectures, our Graph CL framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods. Experiments show that Graph CL achieves state-of-the-art performance in the settings of semi-supervised learning, unsupervised representation learning and transfer learning. It additionally boosts robustness against common adversarial attacks. |
| Researcher Affiliation | Collaboration | 1Texas A&M University, 2University of Texas at Austin, 3University of Science and Technology of China, 4Google Research, Brain Team |
| Pseudocode | Yes | The proposed graph contrastive learning is summarized in Appendix A. (Appendix A: Algorithm 1: Graph Contrastive Learning for pre-training GNNs, Algorithm 2: Random Walk Subgraph Sampling) |
| Open Source Code | Yes | Our codes are available at: https://github.com/Shen-Lab/Graph CL. |
| Open Datasets | Yes | Semi-supervised learning. We first evaluate our proposed framework in the semi-supervised learning setting on graph classification [63, 3] on the benchmark TUDataset [64]. (Table 2: Datasets statistics. Datasets: NCI1, PROTEINS, COLLAB, RDT-B) |
| Dataset Splits | No | Appendix B states: "We randomly split the data into 10 folds and use 9 folds for training and 1 fold for testing. We follow this standard evaluation protocol as in [63, 21]." This describes a train/test split within a cross-validation setup, but no separate validation set split is explicitly detailed for hyperparameter tuning or early stopping. |
| Hardware Specification | No | The paper does not specify the hardware used to run the experiments (e.g., GPU models, CPU types, or memory specifications). |
| Software Dependencies | No | The paper mentions "The experiments are implemented with PyTorch" in Appendix B, but does not provide specific version numbers for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | The default augmentation (dropping, perturbation, masking and subgraph) ratio is set at 0.2. The temperature parameter τ is set to 0.5. For all graph classification tasks, we use the Adam optimizer [69] with an initial learning rate of 0.001, weight decay of 0.0001, and batch size of 128. For semi-supervised learning and unsupervised representation learning, the model is trained for 100 epochs, while for transfer learning, the model is trained for 300 epochs. For adversarial robustness, the models are trained for 500 epochs. The learning rate is decayed by 0.5 every 20 epochs. We use two-layer GCN as the GNN-based encoder and a two-layer MLP as the projection head. Hidden dimensions are 512 for GCN and 2048 for MLP. |