Transformers Generalize DeepSets and Can be Extended to Graphs & Hypergraphs
Authors: Jinwoo Kim, Saeyoon Oh, Seunghoon Hong
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we demonstrate the capability of higher-order Transformers on a variety of tasks including synthetic data, large-scale graph regression, and set-to-(hyper)graph prediction. Specifically, we use a synthetic node classification dataset from Gu et. al. (2020) [11], a molecular graph regression dataset from Hu et. al. (2021) [17], two set-to-graph prediction datasets from Serviansky et. al. (2020) [29], and three hyperedge prediction datasets used in Zhang et. al. (2020) [36]. |
| Researcher Affiliation | Academia | Jinwoo Kim, Saeyoon Oh, Seunghoon Hong School of Computing, KAIST {jinwoo-kim, saeyoon17, seunghoon.hong}@kaist.ac.kr |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | Our implementation is available at https://github.com/jw9730/hot. |
| Open Datasets | Yes | Specifically, we use a synthetic node classification dataset from Gu et. al. (2020) [11], a molecular graph regression dataset from Hu et. al. (2021) [17], two set-to-graph prediction datasets from Serviansky et. al. (2020) [29], and three hyperedge prediction datasets used in Zhang et. al. (2020) [36]. |
| Dataset Splits | Yes | We used training and test sets with chains of length 20 and 200 respectively. ... As test data is unavailable, we report the Mean Absolute Error (MAE) on validation dataset. |
| Hardware Specification | Yes | Plots are shown until each model runs into out-of-memory error on a RTX 6000 GPU with 22GB. |
| Software Dependencies | No | The paper does not explicitly provide a list of software dependencies with specific version numbers (e.g., 'Python 3.8, PyTorch 1.9, and CUDA 11.1'). |
| Experiment Setup | Yes | Details including the datasets and hyperparameters can be found in Appendix A.2. |