Interpretable Graph Networks Formulate Universal Algebra Conjectures
Authors: Francesco Giannini, Stefano Fioravanti, Oguzhan Keskin, Alisia Lupidi, Lucie Charlotte Magister, Pietro Lió, Pietro Barbiero
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The results of our experiments demonstrate that interpretable graph networks: (i) enhance interpretability without sacrificing task accuracy, (ii) strongly generalize when predicting universal algebra s properties, (iii) generate simple explanations that empirically validate existing conjectures, and (iv) identify subgraphs suggesting the formulation of novel conjectures. |
| Researcher Affiliation | Academia | Francesco Giannini CINI, Italy francesco.giannini@unisi.it Stefano Fioravanti Università di Siena, Italy, JKU Linz, Austria Italy stefano.fioravanti@unisi.it Oguzhan Keskin University of Cambridge, UK ok313@cam.ac.uk Alisia Maria Lupidi University of Cambridge, UK aml201@cam.ac.uk Lucie Charlotte Magister University of Cambridge, UK lcm67@cam.ac.uk Pietro Lió University of Cambridge, UK pl219@cam.ac.uk Pietro Barbiero Università della Svizzera Italiana, CH University of Cambridge, UK barbip@usi.ch |
| Pseudocode | Yes | Algorithm 1: Generate dataset of lattice varieties. |
| Open Source Code | Yes | The dataset generator and the datasets are available at https://github.com/fragiannini/AI4UA. |
| Open Datasets | Yes | Using Algorithm 1, we generated the first large-scale AI-compatible datasets of lattices containing more than 29, 000 graphs and the labels of 5 key properties of lattice (quasi-)varieties i.e., modularity, distributivity, semi-distributivity, join semi-distributivity, and meet semi-distributivity, whose definitions can be found in Appendix A. The dataset generator and the datasets are available at https://github.com/fragiannini/AI4UA. |
| Dataset Splits | No | The paper specifies train/test splits but does not explicitly mention a validation set or its size/percentage for reproducibility. It states: "We evaluate generalization under two different conditions: with independently and identically distributed train/test splits, and out-of-distribution by training on graphs up to eight nodes, while testing on graphs with more than eight nodes ( strong generalization (41))." |
| Hardware Specification | Yes | All of our experiments were run on a private machine with 8 Intel(R) Xeon(R) Gold 5218 CPUs (2.30GHz), 64GB of RAM, and 2 Quadro RTX 8000 Nvidia GPUs. |
| Software Dependencies | Yes | For our experiments, we implemented all baselines and methods in Python 3.7 and relied upon open-source libraries such as Py Torch 1.11 (33) (BSD license) and Scikit-learn (34) (BSD license). |
| Experiment Setup | Yes | In practice, we train all models using eight message passing layers and different embedding sizes ranging from 16 to 64. We train all models for 200 epochs with a learning rate of 0.001. For interpretable models, we set the Gumbel-Softmax temperature to the default value of 1 and the activation behavior to "hard," which generates one-hot encoded embeddings in the forward pass, but computes the gradients using the soft scores. For the hierarchical model, we set the internal loss weight to 0.1 (to score it roughly 10% less w.r.t. the main loss). |