Exploring Correlations of Self-Supervised Tasks for Graphs
Authors: Taoran Fang, Wei Chow, Yifei Sun, Kaiqiao Han, Lvbin Ma, Yang Yang
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results indicate that our method significantly outperforms existing methods across various downstream tasks. Our paper aims to provide a fresh understanding of graph self-supervised learning based on task correlations. Specifically, we evaluate the performance of the representations trained by one specific task on other tasks and define correlation values to quantify task correlations. Through this process, we unveil the task correlations between various self-supervised tasks and can measure their expressive capabilities, which are closely related to downstream performance. By analyzing the correlation values between tasks across various datasets, we reveal the complexity of task correlations and the limitations of existing multi-task learning methods. |
| Researcher Affiliation | Collaboration | 1College of Computer Science and Technology, Zhejiang University, Hangzhou, China 2Zhejiang Huayun Information Technology, Hangzhou, China. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at: https: //github.com/zjunet/Graph TCM. |
| Open Datasets | Yes | We employ 6 graph datasets in our experiments, including 4 citation network datasets Cora, Cite Seer, Pub Med, ogbn-arxiv and 2 product network datasets Amazon-Computers, Amazon-Photo. Their detailed introductions can be found in Appendix B.1. Cora, Cite Seer, Pub Med, ogbn-arxiv: These 4 different citation networks are widely used as graph benchmarks (Sen et al., 2008; Hu et al., 2020a). Amazon-Computers, Amazon-Photo: These are 2 product networks (Shchur et al., 2018) from Amazon. |
| Dataset Splits | Yes | Table 6. The statistics of involved datasets. Dataset Nodes Edges Feature Classes Train/Val/Test ... Cora 2708 5429 1433 7 140 / 500 / 1000 ... Cite Seer 3327 4732 3703 6 120 / 500 / 1000 ... Pub Med 19717 44338 500 3 60 / 500 / 1000 ... During the training process, we randomly partition 70% of the correlation values as the training set and the remaining as the validation set. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory specifications) used for running the experiments. It generally refers to GNNs and models but without hardware specifics. |
| Software Dependencies | No | The paper mentions the use of Graph Convolutional Network (GCN) as backbone models but does not list specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions or other libraries). |
| Experiment Setup | Yes | The hyperparameters for the aforementioned Self-supervised Tasks can be found in Table 7. All backbone models for these tasks have utilized the Graph Convolutional Network (GCN) with a linear architecture, and the parameter settings are largely consistent with the original paper. We trained 128-dimensional representations on Cora, Cite Seer, Pub Med, Amazon Computers, and Amazon-Photo, and 32-dimensional representations on ogbn-arxiv. Additionally, for the Graph Comp model, the (learning rate, weight decay) values are set to (5e-4, 0.7) for the Citeseer dataset and (5e-4, 0.5) for the Pubmed dataset. When applying the trained representations to downstream tasks, we exclusively train a linear transformation W Rd d , where d denotes the dimension of the trained representations, and d denotes the dimension required for the downstream tasks. Across all datasets and methods, we maintain consistent hyperparameters, setting the learning rate to 0.001, weight decay to 0.0005, and conducting 300 training epochs. |