Seq2Tens: An Efficient Representation of Sequences by Low-Rank Tensor Projections
Authors: Csaba Toth, Patric Bonnier, Harald Oberhauser
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Code and benchmarks are publically available at https://github.com/tgcsaba/seq2tens. We demonstrate the modularity and flexibility of the above LS2T and its variants by applying it to (i) multivariate time series classification, (ii) mortality prediction in healthcare, (iii) generative modelling of sequential data. In all cases, we take a strong baseline model (FCN and GP-VAE, as detailed below) and upgrade it with LS2T layers. |
| Researcher Affiliation | Academia | Mathematical Institute, University of Oxford {toth, bonnier, oberhauser}@maths.ox.ac.uk |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code and benchmarks are publically available at https://github.com/tgcsaba/seq2tens. |
| Open Datasets | Yes | We consider multivariate time series classification (TSC) on an archive of benchmark datasets collected by Baydogan (2015). We consider the PHYSIONET2012 challenge dataset (Goldberger et al., 2000) for mortality prediction. imputation of Healing MNIST, Sprites, and Physionet 2012. |
| Dataset Splits | Yes | For both models, we conduct a random search for all hyperparameters with 20 samples from a pre-specified search space, and the setting with best validation performance is used for model evaluation on the test set over 5 independent model trains, exactly the same way as it was done in Horn et al. (2020). |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments (e.g., GPU models, CPU types, or cloud instance specifications). |
| Software Dependencies | No | The paper mentions software like 'FCN', 'Res Net', 'LSTM', 'PyTorch', but does not specify version numbers for any of these or other key dependencies. |
| Experiment Setup | Yes | The FCN is a fully convolutional network which stacks 3 convolutional layers of kernel sizes (8, 5, 3) and filters (128, 256, 128) followed by a global average pooling (GAP) layer. These hyperparameters were only subject to hand-tuning on a subset of the datasets, and the values we considered were H, N {32, 64, 128}, M {2, 3, 4} and D {1, 2, 3}, where H, N N is the FCN and LS2T width, resp., while M N is the LS2T order and D N is the LS2T depth. We additionally introduce in both models a Spatial Dropout1D layer after all CNN and LS2T layers with the same tunable dropout rate to mitigate the low SNR of the dataset. |