Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
How Attentive are Graph Attention Networks?
Authors: Shaked Brody, Uri Alon, Eran Yahav
ICLR 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform an extensive evaluation and show that GATv2 outperforms GAT across 12 OGB and other benchmarks while we match their parametric costs. Our code is available at https://github.com/tech-srl/how_attentive_are_ gats.1 |
| Researcher Affiliation | Academia | Shaked Brody Technion EMAIL Uri Alon Language Technologies Institute Carnegie Mellon University EMAIL Eran Yahav Technion EMAIL |
| Pseudocode | No | The paper describes mathematical formulations for GAT and GATv2 (Equations 6 and 7) but does not provide pseudocode or a clearly labeled algorithm block. |
| Open Source Code | Yes | Our code is available at https://github.com/tech-srl/how_attentive_are_ gats.1 |
| Open Datasets | Yes | We used the provided splits of OGB (Hu et al., 2020) and the Adam optimizer. We further compare GATv2, GAT, and other GNNs on four node-prediction datasets from OGB. |
| Dataset Splits | Yes | We tuned hyperparameters according to validation score and early stopping. |
| Hardware Specification | Yes | Our experiments consumed approximately 100 days of GPU in total. We used cloud GPUs of type V100, and we used RTX 3080 and 3090 in local GPU machines. |
| Software Dependencies | No | GATv2 is available as part of the Py Torch Geometric library, the Deep Graph Library, and the Tensor Flow GNN library. While these libraries are mentioned, no specific version numbers are provided for them or any other software dependencies. |
| Experiment Setup | Yes | We tuned the following hyperparameters: number of layers {2, 3, 6}, hidden size {64, 128, 256}, learning rate {0.0005, 0.001, 0.005, 0.01} and sampling method full batch, Graph SAINT (Zeng et al., 2019) and Neighbor Sampling (Hamilton et al., 2017). |