Time Series Kernels based on Nonlinear Vector AutoRegressive Delay Embeddings
Authors: Giovanni De Felice, John Goulermas, Vladimir Gusev
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate excellent performance on a wide range of real-world classification tasks, both in terms of accuracy and speed. This further advances the understanding of RC representation learning models and extends the typical use of the NVAR framework to kernel design and representation of real-world time series data. 4 Experimental evaluation In this section, we present an experimental demonstration of the performance of NVARk. In line with the established literature (Cuturi, 2011; Cuturi & Doucet, 2011; Baydogan & Runger, 2016; Paparrizos et al., 2020), our main evaluation consists of time series classification. In particular, we pair different pre-computed kernel matrices with a Support Vector Machine (SVM) classifier (Steinwart & Christmann, 2008), a popular kernel classification method that lies between simple linear models and more advanced deep learning architectures. The evaluation is performed over 130 UTS datasets from the UCR archive and 19 MTS datasets from the UEA archive (Bagnall A. & E.) (Appendix B.1) and focuses on accuracy, execution time and scalability. |
| Researcher Affiliation | Academia | Giovanni De Felice John Y. Goulermas Vladimir V. Gusev Department of Computer Science University of Liverpool gdefe@liverpool.ac.uk gusev@liverpool.ac.uk |
| Pseudocode | Yes | Algorithm 1 NVAR kernel input MTS list X[1 : N], number of lags k, lag size s, embedding size dr 1: L = {(a, p) | a {1, ..., dx}, p = {1, ..., k}} (dimension a lagged p times) 2: P = {((a, p), (b, q)) | a, b {1, ..., dx}, p, q = {0, ..., k}} (all possible products) 3: samples sample ( dr dx) elements from L P at random 4: for i in [1 : N] do 5: R[i] X[i] 6: for elem in samples do 7: if elem = (a, p) is in L then 8: concatenate lag term: R[i] R[i] || xa t p s[i] 9: end if 10: if elem = ((a, p), (b, q)) is in P then 11: concatenate nonlinear term: R[i] R[i] || xa t p s[i] xb t q s[i] 12: end if 13: end for 14: fit a linear model mapping rt 1[i] to rt[i] 15: vectorize fit parameters to obtain representation θ[i] 16: end for 17: for all representation pairs: Kij RBF(θ[i], θ[j]) output K |
| Open Source Code | Yes | An easy-to-use Python implementation of the NVAR kernel is made publicly available at https://github. com/gdefe/nvark-kernel. |
| Open Datasets | Yes | The evaluation is performed over 130 UTS datasets from the UCR archive and 19 MTS datasets from the UEA archive (Bagnall A. & E.) (Appendix B.1) |
| Dataset Splits | Yes | For the CV loop, we adopt a 10-fold CV with the size of the validation set corresponding to 33% of the train set. |
| Hardware Specification | Yes | Experiments were run on a 32-core AMD Ryzen Threadripper PRO CPU. |
| Software Dependencies | No | The paper mentions software like "Python implementation" and "Matlab implementation" but does not specify any version numbers for Python, Matlab, or any associated libraries/packages (e.g., scikit-learn, PyTorch, TensorFlow, numpy, etc.). |
| Experiment Setup | Yes | This section presents the main building blocks of NVAR models, followed by our proposal of an NVAR-based kernel and a general setting for the hyperparameters that govern it. 3.3 Hyperparameter setting We display here the hyperparameters to construct NVARk and propose heuristic expressions. Number of lags (k) and lag size (s) [...] Polynomial order (n) [...] Embedding dimension ( dr) [...] Linear readout regularization (λridge) [...] RBF lengthscale (γrbf) [...] For the rm ESN, we follow the setting given in Bianchi et al. (2020): number of internal reservoir units R = 800; spectral radius ρ = 0.99; no leakage; percentage of non-zero connectivity within the reservoir states β = 0.25; input scaling ω = 0.15; noise level in the reservoir update ϵt = 0.001; transient dropped states ndrop = 5; number of dimensions after the dimensionality reduction module D = 75 (via tenor-PCA, also proposed in the same paper); favoring comparability with NVARk, we set the readout regularization by the OCRe P method; favoring comparability with NVARk, we set the RBF lengthscale to the median pairwise distance between the representations. |