Energy Transformer
Authors: Benjamin Hoover, Yuchen Liang, Bao Pham, Rameswar Panda, Hendrik Strobelt, Duen Horng Chau, Mohammed Zaki, Dmitry Krotov
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | explore its empirical capabilities using the image completion task, and obtain strong quantitative results on the graph anomaly detection and graph classification tasks. |
| Researcher Affiliation | Collaboration | Benjamin Hoover IBM Research Georgia Tech benjamin.hoover@ibm.com Yuchen Liang Department of CS RPI liangy7@rpi.edu Bao Pham Department of CS RPI phamb@rpi.edu Rameswar Panda MIT-IBM Watson AI Lab IBM Research rpanda@ibm.com Hendrik Strobelt MIT-IBM Watson AI Lab IBM Research hendrik.strobelt@ibm.com Duen Horng Chau College of Computing Georgia Tech polo@gatech.edu Mohammed J. Zaki Department of CS RPI zaki@cs.rpi.edu Dmitry Krotov MIT-IBM Watson AI Lab IBM Research krotov@ibm.com |
| Pseudocode | Yes | H Formal Algorithm for Training and Inference of ET. Algorithm 1: Training and inference pseudocode of ET for image reconstruction task |
| Open Source Code | Yes | The code is available: https://github.com/bhoov/energy-transformer-jax. The code is available: https://github.com/zhuergou/Energy-Transformer-for-Graph-Anomaly-Detection/. The code is available: https://github.com/Lemon-cmd/Energy-Transformer-For-Graph. |
| Open Datasets | Yes | We trained* the ET network on the masked image completion task using Image Net-1k dataset [22]. Amazon and Yelp datasets can be obtained from the DGL library, T-Finance and T-Social can be obtained from [24]. Eight graph datasets from the TUDataset [36] collection are used for experimentation. ET is trained and evaluated with five datasets from GNNBenchmark [37]. |
| Dataset Splits | Yes | For each dataset, either 1% or 40% of the nodes are used for training, and the remaining 99% or 60% are split 1 : 2 into validation and testing sets, see Appendix C for details. |
| Hardware Specification | No | While Tables 9 and 10 mention 'num. of gpu devices' (e.g., 'num. of gpu devices 2' or '4'), no specific GPU models (e.g., NVIDIA A100), CPU models, or other detailed hardware specifications are provided for running the experiments. |
| Software Dependencies | No | The paper mentions software frameworks like 'JAX [49]', 'Flax [50]', 'Optax [60]', and 'Py Torch Geometric [61]' but does not provide specific version numbers for any of these dependencies. |
| Experiment Setup | Yes | Table 4: Hyperparameter, architecture, and data augmentation choices for ET model during Image Net-1k masked training experiments. Table 6: Hyperparameters choice of our method on all the datasets. Table 9: Hyperparameter and architecture choices for ET during TUDataset experiments. Tables 10 and 11 also provide hyperparameters for GNNBenchmark datasets. |