Contrastive Self-supervised Learning for Graph Classification
Authors: Jiaqi Zeng, Pengtao Xie10824-10832
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on various graph classification datasets demonstrate the effectiveness of our proposed methods. The code is available at https://github.com/UCSD-AI4H/Graph_SSL. |
| Researcher Affiliation | Academia | Jiaqi Zeng1 and Pengtao Xie2 1 Department of Computer Science and Engineering, Shanghai Jiao Tong University, China 2 Department of Electrical and Computer Engineering, University of California San Diego, USA Gabyyyyyy@sjtu.edu.cn, p1xie@eng.ucsd.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available at https://github.com/UCSD-AI4H/Graph_SSL. |
| Open Datasets | Yes | We used 5 graph classification datasets1 in the experiments. Each data example consists of a graph and a class label. ... 1Datasets are publicly available at https://ls11-www.cs.tudortmund.de/staff/morris/graphkerneldatasets |
| Dataset Splits | Yes | We randomly split each dataset into three parts: 80% for training, 10% for validation, and 10% for testing. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments (e.g., GPU models, CPU specifications, or memory). |
| Software Dependencies | No | The paper mentions optimizers and decay schedules but does not specify any software libraries or frameworks with their version numbers (e.g., PyTorch 1.x, TensorFlow 2.x, Python 3.x, CUDA version). |
| Experiment Setup | Yes | For CSSL pretraining, the queue size in Mo Co is set as 1024 for the D&D and PROTEINS dataset, and 4096 for the NCI1, NCI109, and Mutagenicity dataset. The Mo Co momentum is set as 0.999 and the temperature τ is set as 0.07. The initial learning rate is searched in {1e 3, 1e 4, 1e 5} and decayed with the cosine decay schedule (Loshchilov and Hutter 2016). We find it beneficial to utilize a small batch size (16 or 32), a small learning rate (1e 5), and train for more epochs (1k 3k). For finetuning the classification model, we search the initial learning rate in {1e 2, 1e 3, 1e 4} and utilize the Adam optimizer (Kingma and Ba 2014) to optimize the model. Following (Zhang et al. 2019), we adopt early stopping based on the validation loss. Specifically, we stop training if the validation loss does not decrease for 100 consecutive epochs. We select the model with the smallest validation loss as the final model. For CSSL-Reg We search the regularization parameter λ in {1, 0.1, 0.01, 0.001, 0.0001}. The Adam optimizer is used and the initial learning rate is searched in {1e 2, 1e 3, 1e 4}. We set the queue size in Moco as 512 for the D&D and PROTEINS dataset, and 2048 for the NCI1, NCI109, and Mutagenicity dataset. The settings of batch size, patience for early stopping, Mo Co momentum, and temperature τ are the same as those in CSSL-Pretrain. Graph Encoder: The dimension of node representation is set to 128. The number of HGP-SL layers is set as 3. The pooling ratio is searched in [0.1, 0.9] and the dropout ratio is searched in [0.0, 0.5]. |