Efficient Contrastive Learning for Fast and Accurate Inference on Graphs
Authors: Teng Xiao, Huaisheng Zhu, Zhiwei Zhang, Zhimeng Guo, Charu C. Aggarwal, Suhang Wang, Vasant G Honavar
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The extensive experiments on widely used real-world benchmarks that show that Graph ECL achieves superior performance and inference efficiency compared to state-of-the-art graph constrastive learning (GCL) methods on homophilous and heterophilous graphs. |
| Researcher Affiliation | Collaboration | 1Penn State University, USA 2IBM T. J. Watson Research Center, USA. |
| Pseudocode | No | No pseudocode or algorithm blocks are explicitly presented in the paper. |
| Open Source Code | Yes | Code is available at: https: //github.com/tengxiao1/Graph ECL. |
| Open Datasets | Yes | We use established benchmarks for homophilic graphs: Cora, Citeseer, Pubmed, Photo, Wiki CS, and Flickr, and for heterophilic graphs: Cornell, Wisconsin, Texas, and Actor. Additionally, we evaluate Graph ECL on large-scale graphs, specifically the heterophilic Snap-patents, and homophilic Ogbn-arxiv and Ogbn-papers100M. In all datasets, we use the standard splits used in prior studies (Zhang et al., 2021a). The dataset details, splits, and statistics are in C.3. All datasets and public splits can found in PyTorch Geometric: https://pytorch-geometric.readthedocs. io/en/latest/modules/datasets.html. |
| Dataset Splits | Yes | We utilize the public split: a fixed 20 nodes from each class for training and another distinct set of 500 and 1,000 nodes for validation and testing, respectively. |
| Hardware Specification | Yes | Our experiments were conducted on a machine equipped with NVIDIA RTX A100 GPUs with 80GB memory. |
| Software Dependencies | No | The paper mentions using PyTorch Geometric and Adam optimizer, but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | For our approach, we explored the following hyperparameter ranges: λ from {0.001, 0.01, 0.1, 0.5, 1}, K from {256, 512, 1024, 2048, 4096}, τ from {0.5, 0.75, 0.99, 1}, and the number of negative pairs M from {1, 5, 10} when negative sampling was used. Furthermore, we tuned the learning rate from the set {1e-3, 5e-3, 1e-4} and the weight decay from the set {0, 1e-4, 3e-4, 1e-6}. |