Unsupervised Model Selection for Time Series Anomaly Detection
Authors: Mononito Goswami, Cristian Ignacio Challu, Laurent Callot, Lenon Minorics, Andrey Kan
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Large-scale experiments on multiple real-world datasets demonstrate that our proposed unsupervised approach is as effective as selecting the most accurate model based on partially labeled data. |
| Researcher Affiliation | Collaboration | Mononito Goswami & Cristian Challu Carnegie Mellon University {mgoswami,cchallu}@cs.cmu.edu Laurent Callot, Lenon Minorics & Andrey Kan Amazon Research {lcallot,avkan}@amazon.com, minorics@amazon.de |
| Pseudocode | No | The paper describes its methods in prose and explains algorithms conceptually but does not include any formal pseudocode blocks or clearly labeled algorithm figures. |
| Open Source Code | Yes | We provide an open-source implementations of our model selection method and anomaly detection models at https://github.com/mononitogoswami/tsad-model-selection. |
| Open Datasets | Yes | All datasets used in this work are available in the public domain. Directions to download the Server Machine Dataset are available at https://github.com/Net Man AIOps/Omni Anomaly, whereas the UCR Anomaly Archive can be downloaded from https://www.cs.ucr.edu/ eamonn/time_series_data_2018/. |
| Dataset Splits | Yes | Specifically, we randomly partition each dataset into selection (20%) and evaluation (80%) sets. Each time-series in the dataset is assigned to one of these sets. The baseline method selects a model based on anomaly labels in the selection set. |
| Hardware Specification | Yes | All our experiments were carried out on an AWS g3.4xlarge EC2 instance with with 16 Intel(R) Xeon(R) CPUs, 122 Gi B RAM and a 8 Gi B GPU. |
| Software Dependencies | Yes | All models and model selection algorithms were trained and built using scikit-learn 1.1.1 (Pedregosa et al., 2011), Py Torch 1.11.0 Paszke et al. (2019) along with Python 3.9.13. |
| Experiment Setup | Yes | We created a pool of 3 k-NN (Ramaswamy et al., 2000), 4 moving average, 4 DGHL (Challu et al., 2022), 4 LSTMVAE (Park et al., 2018) and 4 RNN (Chang et al., 2017) models by varying hyperparameters for each model, for a total 19 combinations. Finally, to efficiently train our models, we sub-sampled all time-series with T > 2560 by a factor of 10. |