Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Principled Data-Driven Decision Support for Cyber-Forensic Investigations
Authors: Soodeh Atefi, Sakshyam Panda, Emmanouil Panaousis, Aron Laszka
AAAI 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our proposed approach on multiple versions of the MITRE ATT&CK dataset, which is a knowledge base of adversarial techniques and tactics based on real-world cyber incidents, and demonstrate that our approach outperforms DISCLOSE in terms of techniques discovered per effort spent. 4 Numerical Evaluation We evaluate our proposed approach numerically on public datasets of real-world cyber incidents. |
| Researcher Affiliation | Academia | 1 University of Houston, 2 University of Greenwich, 3 Pennsylvania State University |
| Pseudocode | Yes | Algorithm 1 Exploration Decision Function and Algorithm 2 MCTS for Forensic Decision Support |
| Open Source Code | Yes | Our implementation and datasets are publicly available.1 https://github.com/Soodeh Atefi/Decision Support-AAAI-23 |
| Open Datasets | Yes | We use the MITRE ATT&CK Enterprise repository (Barnum 2012), which is a public repository of adversarial tactics, techniques, & procedures, referencing realworld cyber incidents in which some of these techniques were used. |
| Dataset Splits | Yes | Since the datasets are relatively small, we use a leave-one-out cross-validation: when evaluating a policy on an incident, we treat all other incidents in our dataset as prior incidents I. |
| Hardware Specification | Yes | For a single decision, the MCTS algorithm takes less than 7 seconds on average using a single core of a 2.4GHz Intel Core i9 CPU, and less than a second using multiple cores. |
| Software Dependencies | No | The paper mentions 'Hyperopt Python library' but does not provide a specific version number for it or any other software component used in the experiments. |
| Experiment Setup | Yes | First, we optimized the hyper-parameters for the k-NN probability estimation (β1, β2) using a grid search... Then, we optimized the hyper-parameters for MCTS using Hyperopt Python library... Note that we optimized the hyper-parameters for datasets separately. |