Geometry Awakening: Cross-Geometry Learning Exhibits Superiority over Individual Structures
Authors: YADONG SUN, Xiaofeng Cao, Yu Wang, Wei Ye, Jingcai Guo, Qing Guo
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results demonstrate that our model-agnostic framework more effectively captures topological graph knowledge, resulting in superior performance of the student models when compared to traditional KD methodologies. and 5 Experiments In this section, we first give the experimental setup and baselines. Then we compare our graph KD framework with some baselines on NC and LP tasks. Hyperparameters and ablation analysis also be given. |
| Researcher Affiliation | Academia | Yadong Sun1 Xiaofeng Cao1, Yu Wang1 Wei Ye2 Jingcai Guo3 Qing Guo4 1School of Artificial Intelligence, Jilin University, China 2College of Electronic and Information Engineering, Tongji University, China 3The Hong Kong Polytechnic University 4CFAR and IHPC, Agency for Science, Technology and Research (A*STAR), Singapore |
| Pseudocode | Yes | Algorithm 1 Cross-Geometric Graph KD Input: Graph G = {V, E}; Pre-trained teacher MT and GEO model; Initialization parameters θ of student. Parameter: Threshold λ; Weight β. Output: Distilled model s parameter θ . |
| Open Source Code | Yes | We submitted our code and datasets, and provided a replication guide. (From NeurIPS Checklist, Question 5) |
| Open Datasets | Yes | We preform NC and LP tasks on citation network datasets Cora [42],Citeseer [43] and Pubmed [44], wikipedia-based article hyperlink network dataset Wiki-CS [45], and Physics part of the Coauthor dataset Co-Physics [46]. and Ogbn-Arxiv (1,166,243 edges, 169,343 nodes) and Ogbn-Proteins (39,561,252 edges, 132,534 nodes) [50]. |
| Dataset Splits | Yes | To ensure fairness, we uniformly apply standard splits (70%/15%/15%) for node classification tasks and standard splits (85%/5%/10%) for link prediction tasks. |
| Hardware Specification | Yes | The running environment includes an Intel Core Intel i7-13700KF CPU with a clock speed of 3.40GHz, boasting 16 cores and 24 threads. A robust NVIDIA Ge Force RTX 4070Ti GPU, featuring 12GB of VRAM, encompasses 7680 CUDA cores. The system is equipped with 16GB of RAM. The operating system is Windows 11, and Python 3.10 serves as the programming language. For deep learning tasks, Py Torch version 1.13 is employed, while CUDA version 12.2 enhances GPU acceleration. Package management is facilitated through the use of Anaconda. For large datasets, Pubmed and Coauthor Physics, experiments were conducted on a high-performance server with the following specifications: 4 Intel Xeon Gold 5220 CPUs running at 2.20GHz, equipped with 72 cores and 144 threads. The system features 4 Quadro RTX 6000 GPUs, each boasting 24GB of VRAM and 4608 CUDA cores. The system boasts 500GB of RAM and runs on Ubuntu 18.04.6. |
| Software Dependencies | Yes | The operating system is Windows 11, and Python 3.10 serves as the programming language. For deep learning tasks, Py Torch version 1.13 is employed, while CUDA version 12.2 enhances GPU acceleration. Package management is facilitated through the use of Anaconda. and runs on Ubuntu 18.04.6. |
| Experiment Setup | Yes | Setups. We preform NC and LP tasks on citation network datasets Cora [42],Citeseer [43] and Pubmed [44], wikipedia-based article hyperlink network dataset Wiki-CS [45], and Physics part of the Coauthor dataset Co-Physics [46]. The student and teacher models are both GCN composed of two hidden layers and one output layer. The hidden layer node dimensions are 8 for the student and 128 for teachers. The model parameters are uniformly initialized using the Xavier s uniform initialization [47] method The optimizer uses Adam [48] or Riemannian Adam [49]. We set the value of k for k-hops subgraphs to 4. and The parameter configurations for NC are detailed in Table 6, while those for LP are delineated in Table 7. |