EXACT: Scalable Graph Neural Networks Training via Extreme Activation Compression
Authors: Zirui Liu, Kaixiong Zhou, Fan Yang, Li Li, Rui Chen, Xia Hu
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 EXPERIMENTS The experiments are designed to answer the following research questions. RQ1: How effective is EXACT in terms of model performance at different compression rates (Section 5.1)? RQ2: Is the training process of deeper GNNs also robust to the noise introduced by EXACT (Section 5.1)? RQ3: How sensitive is EXACT to its key hyperparameters (Appendix I.4)? RQ4: What is the running time overhead of EXACT (Section 5.2)? RQ5: Is the convergence speed of GNNs impacted by EXACT (Appendix I.3)? RQ6: To what extent can EXACT reduce the hardware requirement for training GNNs on large graphs (Section 5.3)? |
| Researcher Affiliation | Collaboration | Zirui Liu1, Kaixiong Zhou1, Fan Yang1, Li Li2, Rui Chen2 , and Xia Hu1 1Rice University, 2Samsung Research America |
| Pseudocode | Yes | Algorithm 1: Forward Pass of the lth GCN layer |
| Open Source Code | Yes | The code is available at https://github.com/warai-0toko/Exact. |
| Open Datasets | Yes | To evaluate the scalability of EXACT, we adopt five common large-scale graph benchmark datasets from different domains. Namely, Reddit, Flickr, Yelp, ogbn-arxiv, and ogbn-products. ... Table 10: Dataset Statistics. (contains URLs for each dataset) |
| Dataset Splits | Yes | We report the test accuracy associated with the highest validation score. |
| Hardware Specification | Yes | To answer RQ4, we compare the training throughput of EXACT with the baseline using a single RTX 3090 (24GB) GPU. ... train a full-batch Graph SAGE on ogbn-products on a single GTX 1080 Ti with 11GB memory |
| Software Dependencies | Yes | Table 9: Package configurations of our experiments. Package Version CUDA 11.1 pytorch sparse 0.6.12 pytorch scatter 2.0.8 pytorch geometric 1.7.2 pytorch 1.9.0 OGB 1.3.1 |
| Experiment Setup | Yes | G. EXPERIMENT SETTINGS ... G.2 MODEL HYPERPARAMETER CONFIGURATIONS OF TABLE 1 AND TABLE 2 ... Table 11: Training configuration of Full-Batch GCN, Graph SAGE, and GAT in Table 1 and Table 2. ... Table 13: Training configuration of Cluster-GCN in Table 3. |