Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Reasoning over Streams of Events with Delayed Effects
Authors: Periklis Mantenoglou, Manolis Pitsikalis, Alexander Artikis
JAIR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Fourth, we present a comprehensive, reproducible empirical evaluation based on large synthetic and real data streams, and complex temporal specifications, including a comparison with related systems. Our evaluation demonstrates that RTEC reasons efficiently over streams of events with delayed effects, verifying the complexity analysis, and outperforms the state-of-the-art by orders of magnitude. |
| Researcher Affiliation | Academia | PERIKLIS MANTENOGLOU , NCSR Demokritos , Greece MANOLIS PITSIKALIS, NCSR Demokritos , Greece ALEXANDER ARTIKIS, NCSR Demokritos & University of Piraeus, Greece. Authors Contact Information: Periklis Mantenoglou, orcid: 0009-0002-3275-1522, EMAIL, NCSR Demokritos , Greece; Manolis Pitsikalis, orcid: 0000-0003-2959-2022, EMAIL, NCSR Demokritos , Greece; Alexander Artikis, orcid: 0000-0001-6899-4599, EMAIL, NCSR Demokritos & University of Piraeus, Greece. |
| Pseudocode | Yes | Algorithm 1 process CDComponent. Algorithm 2 eval FI. Algorithm 3 find Attempt. Algorithm 4 compute Attempts. Algorithm 5 initiated At Cyclic(F = V, T). Algorithm 6 process CDComponent. Algorithm 7 eval FT. |
| Open Source Code | Yes | First, we present RTEC , i.e., an open-source1, formal computational framework for reasoning over streams of events with delayed effects. 1https://github.com/aartikis/rtec. The code of RTEC is publicly available2. We provide the code for reproducing our experiments in the form of a Docker image, available in a github repository5. |
| Open Datasets | Yes | Our experiments are reproducible; the event descriptions of all applications and the corresponding datasets are available with the code of RTEC 2. A description of both datasets may be found in (Pitsikalis et al. 2019). |
| Dataset Splits | No | The paper discusses using 'synthetic and real data streams' and mentions 'streams of increasing size' in experimental analysis, but it does not provide specific training/test/validation dataset splits or cross-validation details for reproducibility. |
| Hardware Specification | Yes | We used a single core of a desktop PC running Ubuntu 20.04, with Intel Core i7-4770 CPU @3.40GHz and 16GB RAM. |
| Software Dependencies | No | The paper mentions 'Ubuntu 20.04' as the operating system for the desktop PC. However, it does not list 'multiple key software components with their versions' for the RTEC framework itself, beyond the operating system. It generally states, 'We ran each system using the programming language (version) recommended by its developers,' which is not specific enough for the software dependencies of their own methodology. |
| Experiment Setup | Yes | The window size and the step of RTEC were set to 10 time-points. [...] The window size ranged from 2 to 16 hours and the step was set to 2 hours. |