SVP-T: A Shape-Level Variable-Position Transformer for Multivariate Time Series Classification
Authors: Rundong Zuo, Guozhong Li, Byron Choi, Sourav S Bhowmick, Daphne Ngar-yin Mah, Grace L.H. Wong
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method on all UEA MTS datasets. SVP-T achieves the best accuracy rank compared with several competitive state-of-the-art methods. Furthermore, we demonstrate the effectiveness of the VP-layer and the VPbased self-attention mechanism. Finally, we present one case study to interpret the result of SVP-T. |
| Researcher Affiliation | Academia | 1 Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR 2 School of Computing Engineering, Nanyang Technological University, Singapore 3 Department of Geography, Hong Kong Baptist University, Hong Kong SAR 4 Department of Medicine and Therapeutics, The Chinese University of Hong Kong, Hong Kong SAR |
| Pseudocode | No | The paper describes the methodology in narrative text and equations but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement or link for open-source code availability for the described methodology. |
| Open Datasets | Yes | We evaluate our method on a well-known benchmark of MTS, the UEA archive, which is consists of 30 different datasets. Details of these datasets can be found in (Bagnall et al. 2018). |
| Dataset Splits | Yes | We set a fixed random seed for reproducibility. For consistency of presentation, we follow the results of (Gao et al. 2022) to keep three decimal places. As Table 1 shown, the overall accuracy of SVP-T outperforms all the related methods. Specifically, the average rank of SVP-T is 4.017, which is the best among 13 methods. Meanwhile, the gap in terms of average rank between SVPT and the runner-up, Mini Rocket, is about 1, which shows a clear lead considering that the average ranks of Mini Rocket and RLPAM are nearly the same (less than 0.1 difference). For the number of top-1 accuracy, we find that SVP-T is slightly lower than RLPAM and Mini Rocket. However, the number of top-3 accuracies and the number of top-5 accuracies of SVP-T are both higher than all of the other methods, which shows the results of SVP-T are more robust. Also, the performance of RLPAM relies on the quality of its univariate cluster sequence and Mini Rocket needs random kernels for transformation. In terms of 1-to-1 comparison with other methods, SVP-T wins/draws on at least 18 out of 30 datasets. For the Friedman and Wilcoxon test, we set the significant level to α = 0.05 as (Bagnall et al. 2018; Li et al. 2021). The statistical significance is p 0.05, which confirms there is a significant difference among the 13 methods. The p-values of SVP-T to all methods are less than 0.05, which indicates the results are statistically significant except for Shape Net, Rocket, Mini Rocket, and RLPAM. |
| Hardware Specification | Yes | All the experiments are implemented on a machine with one Xeon Gold 6226 CPU @ 2.70GHz and one NVIDIA Tesla V100S. |
| Software Dependencies | Yes | Python 3.8, and Pytorch 1.10.0 are used to build and train our model. |
| Experiment Setup | Yes | We set α = 1.5 and β = 0 in Formula 9. Since the benchmark datasets are highly heterogeneous, as well as the MTS data in nature, we follow the previous work (Zerveas et al. 2021), that splits the training set into two parts, 80% 20%. Then, we take the 20% part as the validation set to tune the hyperparameters. When the hyperparameters are fixed, we train our model on the whole training set and finally, evaluate it on the official test set. Our tuned hyperparameters of all datasets are shown in Appendix A.1. We adopt batch normalization, instead of layer normalization, for better performance on time series applications (Zerveas et al. 2021). |