Evaluating Robustness and Uncertainty of Graph Models Under Structural Distributional Shifts
Authors: Gleb Bazhenov, Denis Kuznedelev, Andrey Malinin, Artem Babenko, Liudmila Prokhorenkova
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our experiments, we thoroughly evaluate the proposed distributional shifts and show that they can be quite challenging for existing graph models. |
| Researcher Affiliation | Collaboration | Gleb Bazhenov HSE University, Yandex Research Denis Kuznedelev Yandex Research, Skoltech Andrey Malinin Isomorphic Labs Artem Babenko Yandex Research, HSE University Liudmila Prokhorenkova Yandex Research |
| Pseudocode | No | The paper describes algorithms and methods but does not include any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | whereas the remaining methods are implemented in our custom experimental framework and can be found in our repository.5 (Footnote 5: Link to our Git Hub repository) |
| Open Datasets | Yes | For our experiments, we pick the following seven homophilous datasets that are commonly used in the literature: three citation networks, including Cora ML, Cite Seer [26, 9, 8, 31], and Pub Med [28], two co-authorship graphs Coauthor Physics and Coauthor CS [32], and two co-purchase datasets Amazon Photo and Amazon Computer [25, 32]. Moreover, we consider OGB-Products, a large-scale dataset from the OGB benchmark. |
| Dataset Splits | Yes | The half of nodes with the smallest values of σi are considered to be ID and split into Train, Valid-In, and Test-In uniformly at random in proportion 30% : 10% : 10%. The second half contains the remaining OOD nodes and is split into Valid-Out and Test-Out in the ascending order of σi in proportion 10% : 40%. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models or types of computing resources used for experiments. |
| Software Dependencies | No | The paper mentions software components like GCN [16], SAGE [11], and Adam optimizer [14], but it does not specify their version numbers for reproducibility. |
| Experiment Setup | Yes | The hidden dimension of our baseline architecture is 256, and the dropout between hidden layers is p = 0.2. We exploit a standard Adam optimizer [14] with a learning rate of 0.0003 and a weight decay of 0.00001. |