Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication
Authors: Ajay Kumar Jaiswal, Shiwei Liu, Tianlong Chen, Ying Ding, Zhangyang Wang
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our extensive experiments across many real-world small and large graphs, illustrate the effectiveness of our approach and point towards a promising orthogonal direction for GNN scaling. |
| Researcher Affiliation | Academia | Ajay Jaiswal 1 Shiwei Liu 1 Tianlong Chen 1 Ying Ding 1 Zhangyang Wang 1 *Equal contribution 1University of Texas at Austin. Correspondence to: Ajay Jaiswal <ajayjaiswal@utexas.edu>. |
| Pseudocode | Yes | Algorithm 1 Greedy Interpolation Soup Procedure Algorithm 2 Model soup with Graph Sampling Algorithm 3 Model soup with Graph Partitioning |
| Open Source Code | Yes | Codes are available at: https://github. com/VITA-Group/graph_ladling. |
| Open Datasets | Yes | For our small-scale experiments, we use three standard citation network datasets: Cora, Citeseer, and Pubmed, while our large-scale experiments are conducted with four popular large-scale open benchmarks: Flickr, Reddit, OGBN-Ar Xiv, and OGBN-products (Appendix A). |
| Dataset Splits | Yes | For our evaluation on our chosen datasets, we have closely followed the data split settings and metrics reported by the recent benchmark (Duan et al., 2022; Chen et al., 2021). ... Table 9. The searched optimal hyperparameters for all tested methods for data-centric soup. Split: 0.50/0.25/0.25 Split: 0.66 / 0.10 / 0.24 Split: 0.10 / 0.02 / 0.88 |
| Hardware Specification | Yes | Our experiments are conducted on two GPU servers equipped with RTX A6000 and RTX 3090 GPUs. |
| Software Dependencies | No | The paper does not list specific version numbers for software dependencies used in the experiments. It mentions PyTorch Geometric in Appendix A for Flickr and Reddit datasets, but without a version number. |
| Experiment Setup | Yes | The hyperparameters for soup ingredients corresponding to different datasets training are selected via our built-in efficient parameter sweeping functionalities from pre-defined ranges nearby our optimal setting in Appendix B. ... we have used 50 candidate ingredients for small-scale graphs and 30 candidate ingredients for large-scale graphs experiments and our model soup is prepared using interpolation hyperparameter α [0 1] with a step size of 0.01 in Algorithm 1. ... Table 9. The searched optimal hyperparameters for all tested methods for data-centric soup. LR: 0.0001, WD: 0.0001, DP: 0.5, EP: 50, HD: 512, L: 4, BS: 1000 |