Liquid Structural State-Space Models
Authors: Ramin Hasani, Mathias Lechner, Tsun-Hsuan Wang, Makram Chahine, Alexander Amini, Daniela Rus
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive empirical evaluation, we show that Liquid-S4 consistently leads to better generalization performance compared to all variants of S4, CNNs, RNNs, and Transformers across many time-series modeling tasks. In this section, we present an extensive evaluation of Liquid-S4 on sequence modeling tasks with very long-term dependencies and compare its performance to a large series of baselines... |
| Researcher Affiliation | Academia | Ramin Hasani CSAIL, MIT Mathias Lechner CSAIL, MIT Tsun-Hsuan Wang CSAIL, MIT Makram Chahine CSAIL, MIT Alexander Amini CSAIL, MIT Daniela Rus CSAIL, MIT |
| Pseudocode | Yes | Algorithm 1 LIQUID-S4 KERNEL |
| Open Source Code | Yes | Code: https://github.com/raminmh/liquid-s4. |
| Open Datasets | Yes | We first evaluate Liquid-S4 s performance on the well-studied Long Range Arena (LRA) benchmark (Tay et al., 2020b)... We then report Liquid-S4 s performance... on the BIDMC Vital Signals dataset (Pimentel et al., 2016; Goldberger et al., 2000). We also experiment with the s CIFAR dataset... Finally, we perform Raw Speech Command (SC) recognition... |
| Dataset Splits | Yes | Table 1: Performance on Long Range Arena Tasks. Numbers indicate validation accuracy (standard deviation). Table 2: Performance on BIDMC Vital Signs dataset. Numbers indicate RMSE on the test set. Table 3: Performance on s CIFAR. Numbers indicate Accuracy (standard deviation). Table 4: Performance on Raw Speech Command dataset with Full 35 Labels.Numbers indicate Accuracy on test set. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Py Keops package' but does not specify its version number, nor does it list other key software components with versions. |
| Experiment Setup | Yes | Table 5: Hyperparameters for obtaining best performing models. BN= Batch normalization, LN = Layer normalization, WD= Weight decay. (followed by a table with specific values for Depth, Features, H State Size, Norm, Pre-norm, Dropout, LR, Batch Size, Epochs, WD for various tasks). |