A-NeSI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inference
Authors: Emile van Krieken, Thiviyan Thanapalasingam, Jakub Tomczak, Frank van Harmelen, Annette Ten Teije
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that A-NESI is the first end-to-end method to solve three neurosymbolic tasks with exponential combinatorial scaling. Finally, our experiments show that A-NESI achieves explainability and safety without a penalty in performance. |
| Researcher Affiliation | Academia | Emile van Krieken University of Edinburgh Vrije Universiteit Amsterdam Emile.van.Krieken@ed.ac.uk Thiviyan Thanapalasingam University of Amsterdam Jakub M. Tomczak Eindhoven University of Technology Frank van Harmelen, Annette ten Teije Vrije Universiteit Amsterdam |
| Pseudocode | Yes | Algorithm 1 Compute inference model loss; Algorithm 2 A-NESI training loop |
| Open Source Code | Yes | Code is available at https://github.com/HEmile/a-nesi. |
| Open Datasets | Yes | Like [36, 37], we take the MNIST [30] dataset and use each digit exactly once to create data. |
| Dataset Splits | Yes | We performed hyperparameter tuning on a held-out validation set by splitting the training data into 50.000 and 10.000 digits, and forming the training and validation sets from these digits. |
| Hardware Specification | Yes | We used Nvidia RTX A4000s GPUs and 24-core AMD EPYC-2 (Rome) 7402P CPUs. For our experiments, we used the DAS-6 compute cluster [6]. We also thank SURF (www.surf.nl) for its support in using the Lisa Compute Cluster. |
| Software Dependencies | No | The paper mentions using 'ADAM' [25] and 'PyTorch library Storchastic [53]' but does not specify version numbers for these software components. |
| Experiment Setup | Yes | We performed hyperparameter tuning on a held-out validation set by splitting the training data into 50.000 and 10.000 digits, and forming the training and validation sets from these digits. We ran each experiment 10 times to estimate average accuracy, where each run computes 100 epochs over the training dataset. We give the final hyperparameters in Table 4. We use this same set of hyperparameters for all N. # of samples refers to the number of samples we used to train the inference model in Algorithm 1. For simplicity, it is also the beam size for the beam search at test time. The hidden layers and width refer to MLP that computes each factor of the inference model. There is no parameter sharing. |