Linear Dynamical Systems as a Core Computational Primitive
Authors: Shiva Kaul
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present our approach in a bottom-up fashion. Then, we empirically evaluate it on artificial and real datasets. LDS achieve state-of-the-art performance on the copy memory problem. LDStack can be substantially faster than traditional RNNs, while achieving competitive accuracy. Finally, we offer guidance on how our constructions could be improved in future work. |
| Researcher Affiliation | Academia | Shiva Kaul Computer Science Department Carnegie Mellon University Pittsburgh, PA 15213 skkaul@cs.cmu.edu |
| Pseudocode | Yes | Figure 1: Summary of how reachable SIMO LDS, with spectral parameterizations, can be used as a fast layer in a neural network. Also consider the hinge parameterization in the appendix. Martin and Cundy [2018] implemented the PLR algorithm in CUDA; we extend it for complex inputs. [Includes numbered steps: 1. Initialize real variables... 2. Given a sequence of inputs... 3. (Optional). Convert st...] Also, Proposition 2 refers to |
| Open Source Code | No | LDStack (prototype code in both Python and CUDA) is always faster than unfused RNNs. (This implies code exists, but there is no explicit statement of public release or a link to a repository for the code described in the paper.) |
| Open Datasets | Yes | Copy memory problem [Arjovsky et al., 2016, Hochreiter and Schmidhuber, 1997]. [...] Sequential permuted MNIST. [...] Adding problem [Arjovsky et al., 2016, Hochreiter and Schmidhuber, 1997]. |
| Dataset Splits | No | The paper mentions using specific datasets/problems but does not explicitly provide details about training, validation, and test splits (e.g., percentages, sample counts, or explicit standard splits). |
| Hardware Specification | No | LDStack (prototype code in both Python and CUDA). (This implies the use of NVIDIA GPUs due to CUDA, but no specific GPU model or other hardware specifications like CPU, memory, or cloud instances are mentioned). |
| Software Dependencies | No | LDStack (prototype code in both Python and CUDA). (This mentions Python and CUDA, and |
| Experiment Setup | Yes | We use n = 160, which results in just 3380 parameters, including C Cn n. [...] We compare an n = 384 SIMO LDS having ~16,500 parameters to an n = 128 LSTM having ~68,000 parameters, as well as an n = 128 tanh RNN having ~18,000. [...] We use LDStack with state size n = 32, depth = 2, and r = 6 projections. |