Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Precision and Recall for Time Series
Authors: Nesime Tatbul, Tae Jun Lee, Stan Zdonik, Mejbah Alam, Justin Gottschlich
NeurIPS 2018 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we present an experimental study of our range-based model applied to results of state-of-the-art AD algorithms over a collection of diverse time series datasets. |
| Researcher Affiliation | Collaboration | Nesime Tatbul Intel Labs and MIT EMAIL Tae Jun Lee Microsoft EMAIL Stan Zdonik Brown University EMAIL Mejbah Alam Intel Labs EMAIL Justin Gottschlich Intel Labs EMAIL |
| Pseudocode | Yes | Figure 2: Example definitions for ω() and δ() functions. (a) Overlap size function ω(Anomaly Range, Overlap Set, δ) (b) Positional bias function δ(i, Anomaly Length) |
| Open Source Code | No | The paper provides links to third-party tools (NAB Data Corpus [32], Paranom tool [15], Luminol [28]) used in their experiments, but does not state that the source code for their own proposed model or implementations is open-source or available. |
| Open Datasets | Yes | The real datasets are taken from the NAB Data Corpus [32], whereas the synthetic datasets are generated by the Paranom tool [15]. ... [32] Numenta. NAB Data Corpus. https://github.com/numenta/NAB/tree/master/data/, 2017. ... [15] J. Gottschlich. Paranom: A Parallel Anomaly Dataset Generator. https://arxiv.org/abs/1801. 03164/, 2018. |
| Dataset Splits | No | The paper states: "For training and testing, we carefully partition each dataset..." but does not specify details about a validation split or its size/methodology. |
| Hardware Specification | Yes | System: All experiments were run on a Windows 10 machine with an Intel R Core TM i5-6300HQ processor running at 2.30 GHz with 4 cores and 8 GB of RAM. |
| Software Dependencies | No | The paper mentions software like "TensorFlow-implemented version of LSTM-AD", "Greenhouse [26]", and "Luminol [28]" but does not provide specific version numbers for any of these software dependencies. |
| Experiment Setup | Yes | Unless otherwise stated, we use the following default parameter settings for computing our Recall T and Precision T equations: α = 0, γ() = 1, ω() is as in Figure 2a, and δ() returns flat bias as in Figure 2b. |