Position: Graph Foundation Models Are Already Here
Authors: Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma, Tong Zhao, Neil Shah, Mikhail Galkin, Jiliang Tang
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | Position: Graph Foundation Models Are Already Here. In this paper, we present a vocabulary perspective to clearly state the position of the GFM. In particular, we attribute the existing success of primitive GFMs to the suitable vocabulary construction guided by the particular transferability principle on graphs in Section 2. A comprehensive review of the graph transferability principles and corresponding actionable steps is illustrated in Section 3, serving us the principle for future vocabulary construction and the GFM design. In Section 4, we discuss the potential for building the GFM following neural scaling laws from several perspectives (1) building and training vocabulary from scratch, and (2) leveraging existing LLM. Finally, we introduce more insights and open questions to inspire constructive discussions on the GFMs in Section 5. |
| Researcher Affiliation | Collaboration | 1Michigan State University 2Universit e de Montr eal 3Mila Qu ebec AI Institute 4Rensselaer Polytechnic Institute 5Snap Inc. 6Intel Labs. |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | No | The paper does not provide explicit statements or links for its own source code. It mentions 'All relevant resources with GFM design can be found here.' in the abstract without a link, and Appendix A discusses 'open-source graph datasets' which refers to external datasets, not the authors' code. |
| Open Datasets | Yes | Table 1. A collection of datasets together with their URL and descriptions to support larger-scale pre-training. Examples include: TU-DATASET (MORRIS ET AL., 2020) https://chrsmrrs.github.io/datasets/, NETWORKREPOSITORY (ROSSI & AHMED, 2015) https://networkrepository.com/, OPEN GRAPH BENCHMARK (HU ET AL., 2020) https://ogb.stanford.edu/. |
| Dataset Splits | No | The paper is a position paper and review, discussing principles and concepts rather than conducting its own experiments with data. Therefore, it does not provide specific training/validation/test dataset splits. |
| Hardware Specification | No | The paper does not specify any hardware details used for computational work. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies. |
| Experiment Setup | No | This paper is a position paper and review, and does not present its own experiments. Therefore, it does not provide specific experimental setup details or hyperparameters. |