Uncertainty for Active Learning on Graphs
Authors: Dominik Fuchsgruber, Tom Wollschläger, Bertrand Charpentier, Antonio Oroz, Stephan Günnemann
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We propose the first extensive study of Uncertainty Sampling for node classification: (1) We benchmark Uncertainty Sampling beyond predictive uncertainty and highlight a significant performance gap to other Active Learning strategies. (2) We develop ground-truth Bayesian uncertainty estimates in terms of the data generating process and prove their effectiveness in guiding Uncertainty Sampling toward optimal queries. We confirm our results on synthetic data and design an approximate approach that consistently outperforms other uncertainty estimators on real datasets. |
| Researcher Affiliation | Academia | 1School of Computation, Information and Technology, Technical University of Munich, Germany 2Munich Data Science Institute, Germany. |
| Pseudocode | No | No pseudocode or clearly labeled algorithm blocks were found. The paper describes methods in text. |
| Open Source Code | Yes | 1Find our code at cs.cit.tum.de/daml/graph-active-learning/ |
| Open Datasets | Yes | We evaluate AL on five common citation benchmark datasets for node classification: Cora ML (Bandyopadhyay et al., 2005), Citeseer (Sen et al., 2008; Giles et al., 1998), Pub Med (Namata et al., 2012) as well as the co-purchase graphs Amazon Photos and Amazon Computers (Mc Auley et al., 2015). |
| Dataset Splits | Yes | We also perform early stopping on the validation loss with a patience of 100 iterations. |
| Hardware Specification | Yes | (i) Xeon E5-2630 v4 CPU @ 2.20GHz with a NVIDA GTX 1080TI GPU and 128 GB of RAM. (ii) AMD EPYC 7543 CPU @ 2.80GHz with a NVIDA A100 GPU and 128 GB of RAM . |
| Software Dependencies | No | We implement our models in Py Torch and Py Torch Geometric. Specific version numbers for software libraries are not provided. |
| Experiment Setup | Yes | As hyperparameter tuning may be unrealistic in AL (Regol et al., 2020), we do not finetune them on a validation set. ... Specifically, we chose the following values: Model Hidden Dimensions Learning Rate Weight Decay ... Table 3: Hyperparameters of different GNN backbones |