Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
NeSyA: Neurosymbolic Automata
Authors: Nikolaos Manginas, George Paliouras, Luc De Raedt
IJCAI 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We first benchmarked Ne Sy systems on a synthetic task, which allowed us to control the complexity. In particular, we used the domain introduced as a running example, in which a sequence of images must be classified according to a temporal pattern... We use the same setup for generating a training and a testing set. The neural component of all systems is a CNN... We benchmarked against DEEPSTOCHLOG [Winters et al., 2022] and DEEPPROBLOG [Manhaeve et al., 2018] in terms of scalability and with FUZZYA [Umili et al., 2023b] both in terms of scalability and accuracy. Figure 4 shows the comparison... Table 1: Accuracy results on a test set, and timings (in minutes)... In our second experiment we compared NESYA against pure neural solutions on an event recognition task from the CAVIAR benchmark dataset... The task was to recognize events... We present three methods; NESYA a CNN-LSTM and a CNN-Transformer... The data consists of 8 training and 3 testing sequences... We use the macro F1 score for evaluation of all models. |
| Researcher Affiliation | Academia | 1Department of Computer Science and Leuven.AI, KU Leuven, Belgium 2Institute of Informatics and Telecommunications, NCSR Demokritos , Greece 3Centre for Applied Autonomous Sensor Systems (AASS), Orebro University, Sweden |
| Pseudocode | No | The paper describes the inference and learning processes using mathematical formulations and recursive definitions (e.g., α-recursion and WMC calculation) but does not include any explicitly labeled pseudocode or algorithm blocks with structured steps. |
| Open Source Code | Yes | Code is available at: https://github.com/nmanginas/nesya. |
| Open Datasets | Yes | In our second experiment we compared NESYA against pure neural solutions on an event recognition task from the CAVIAR benchmark dataset2. The task was to recognize events performed by pairs of people in raw video data. We focused on two of the events present in CAVIAR, namely moving and meeting, which appear more frequently in the data, and a third no event class... 2https://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/ |
| Dataset Splits | Yes | For each pattern, we generated 100 random trajectories which satisfy the pattern (positive) and 100 negative ones. We use the same setup for generating a training and a testing set. The data consists of 8 training and 3 testing sequences. |
| Hardware Specification | Yes | All experiments were run on a machine with an AMD Ryzen Threadripper PRO 3955WX 16-Core processor, 128GB of RAM, and 2 NVIDIA RTX A6000 with 50GB of VRAM of which only one was utilized. |
| Software Dependencies | Yes | All experiments were implemented in Pytorch and Python 3.11. |
| Experiment Setup | Yes | Both systems are trained with a learning rate of 0.001 following [Umili et al., 2023b]... For all systems training is stopped by monitoring the training loss with a patience of 10 epochs. The Transformer architechture has 3 attention heads per layer, 4 layers and an hidden state dimensionality of 129 (same with the input dimensions). LSTM with a single layer and a 128 dimensional hidden state. |