Learning where and when to reason in neuro-symbolic inference
Authors: Cristina Cornelio, Jan Stuehmer, Shell Xu Hu, Timothy Hospedales
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that our method outperforms the state of the art on visual-Sudoku, and can also benefit visual scene graph prediction. Furthermore, it can improve the performance of existing neuro-symbolic systems that lack our explicit reasoning during inference. ... We considered a total of four Sudoku datasets, summarised in Table1. |
| Researcher Affiliation | Collaboration | Samsung AI, Cambridge {c.cornelio,shell.hu,t.hospedales}@samsung.com Heidelberg Institute for Theoretical Studies and Karlsruhe Institute of Technology {jan.stuehmer}@h-its.org.com |
| Pseudocode | No | Not found. The paper describes the pipeline and learning paradigm verbally and with equations, but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available at: https://github.com/corneliocristina/NASR. |
| Open Datasets | Yes | We considered a total of four Sudoku datasets, summarised in Table1. Three were drawn from an online repository online4: 1) big kaggle contains 100 000 puzzles. It is a subset of a bigger dataset containing 1 million boards hosted on Kaggle5 targeting ML models. ... We converted these symbolic datasets into images using MNIST digits Le Cun & Cortes (2010). ... Finally, we considered the dataset released with Sat Net(Wang et al., 2019) (denoted as satnet data). |
| Dataset Splits | Yes | For training the Mask-Predictor we generate a synthetic dataset Dmp = {yi n, mi}N i=1 where: yn is a symbolic solution instance with the addition of noise that violates the domain-knowledge constraints; and m is the corresponding masking solution. ... The GQA dataset ... train/val/test split that we used are provided in table 6. ... train 66078 ... val (all shots) 4903 ... test (all shots) 10055 |
| Hardware Specification | No | Not found. The paper does not explicitly describe the specific hardware (e.g., CPU/GPU models, memory) used to run its experiments. It mentions 'AWS p3.8xlarge with V100 GPUs' in the context of an example of what could be computationally expensive, not as their own experimental setup. |
| Software Dependencies | Yes | For logic based constraints, classical symbolic reasoners can be used, such as Prolog engines (e.g. SWI-Prolog swi), probabilistic logic engines (e.g. Prob Log, Raedt et al.2007), python libraries that mimic symbolic reasoning engines (e.g. Py Swip, Tekol & contributors2020), theorem provers (e.g. Vampire prover Riazanov & Voronkov2002), etc.; For arithmetic constraints, constraints-solvers can be used (e.g. ILP or MILP solvers), general mathematical tools (e.g. Mathematica mat) or ad-hoc brute force algorithms that exhaustively explore the symbolic solution search space. ... Wolfram Mathematica. https://www.wolfram.com. Version: 12. SWI-prolog. https://www.swi-prolog.org. Version: 8.3.3. |
| Experiment Setup | Yes | The Perception model is trained to minimize the negative log likelihood loss (with log softmax as output layer), and is optimized via the ADADELTA optimizer. ... The Solver NN model is trained to minimize the Binary Cross Entropy with logits loss, and is optimized via Adam optimizer with weight decay. ... We trained the transformer models for 200 epochs, while the Perception models for 100 epochs. RL refinement is trained for 20 epochs. |