Transformation of ReLU-based recurrent neural networks from discrete-time to continuous-time
Authors: Zahra Monfared, Daniel Durstewitz
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Here we show how to perform such a translation from discrete to continuous time for a particular class of Re LU-based RNN. We prove three theorems on the mathematical equivalence between the discrete and continuous time formulations under a variety of conditions, and illustrate how to use our mathematical results on different machine learning and nonlinear dynamical systems examples. ... We will exemplify these results on a couple of machine learning and DS models, including an ODE solution to the well-known addition problem (Hochreiter & Schmidhuber, 1997), limit cycle and chaotic dynamics, and on a PLRNN inferred from empirical time series (human functional magnetic resonance imaging [f MRI] data; (Koppe et al., 2019)). |
| Researcher Affiliation | Academia | 1Department of Theoretical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany 2Faculty of Physics and Astronomy, Heidelberg University, Heidelberg, Germany. |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | Yes | Matlab code for all these examples is available at github.com/Durstewitz Lab/cont PLRNN. |
| Open Datasets | Yes | Example 1. Consider a discrete-time PLRNN emulation of the nonlinear van-der-Pol oscillator, derived by training a discrete PLRNN with M = 10 units on time series generated by the van-der-Pol equations (taken from (Koppe et al., 2019), provided online at github.com/Durstewitz Lab). ... Example 4. As a final example we applied the continuous-time transform developed here to a PLRNN inferred from empirical time series, namely human f MRI data. ... Details on the experimental procedure and task and on PLRNN training can be found in (Koppe et al., 2019). |
| Dataset Splits | No | The paper does not provide explicit training/validation/test dataset splits. |
| Hardware Specification | No | No specific hardware used for experiments is mentioned. |
| Software Dependencies | No | The paper mentions "Matlab code" but does not specify version numbers or other software dependencies with versions. |
| Experiment Setup | Yes | Example 1. Consider a discrete-time PLRNN emulation of the nonlinear van-der-Pol oscillator, derived by training a discrete PLRNN with M = 10 units... Example 2. A simple discrete-time 2-unit PLRNN which (approximately) solves this task is the one with parameters A = [0.99 0; 0 0.99], W = [0 0; 0 0.01], h = [0; 0]... Inputs were accommodated by adding a term Cst to eq. (6), with C = [0 0; 1 1]. ... Example 3. As an example for a system with chaotic dynamics we chose a PLRNN emulation (M = 10)... Example 4. A discrete PLRNN with M = 10 latent states was used for this purpose... |