Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems
Authors: Manuel Brenner, Florian Hess, Jonas M Mikhaeil, Leonard F Bereska, Zahra Monfared, Po-Chen Kuo, Daniel Durstewitz
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that the dendritically expanded PLRNN achieves better reconstructions with fewer parameters and dimensions on various dynamical systems benchmarks and compares favorably to other methods, while retaining a tractable and interpretable structure. ... We employ two frameworks for training the system, one combining back-propagation-through-time (BPTT) with teacher forcing, and another based on fast and scalable variational inference. ... Section 4. Experiments |
| Researcher Affiliation | Academia | 1Dept. of Theoretical Neuroscience, Central Institute of Mental Health, Mannheim, Germany 2Faculty of Physics and Astronomy, Heidelberg University, Germany 3University of Amsterdam, Netherlands 4National Taiwan University, Taiwan. |
| Pseudocode | No | The paper describes methods and derivations using mathematical equations and prose, but it does not include any distinct pseudocode blocks or algorithm listings. |
| Open Source Code | Yes | All code created in here is available at https://github. com/Durstewitz Lab/dend PLRNN. |
| Open Datasets | Yes | The Lorenz-63 and Lorenz-96 systems were simulated using scipy.integrate, while for the bursting neuron and neural population model we used the code provided in Schmidt et al. (2021) and Landau & Sompolinsky (2018), respectively. ... EEG Dataset Electroencephalogram (EEG) data were taken from a study by (Schalk et al., 2000) available at https://physionet.org/content/eegmmidb/1.0.0/. ... ECG Dataset Electrocardiogram (ECG) time series were taken from the PPG-Da Li A dataset (Reiss et al., 2019). ... Wilson-Cowan model... For simulating the model, we used the implementation provided at https://github.com/Open Source Brain/ Wilson Cowan. |
| Dataset Splits | No | The paper mentions training and test sets and performs hyperparameter tuning via grid search, which implies the use of a validation set. However, it does not provide explicit details on the training/validation/test split percentages or sample counts for all datasets in a reproducible manner. |
| Hardware Specification | No | The paper mentions: "For instance, for a PLRNN trained on the Lorenz-63 system (see sect. 4), we exactly located all fixed points in less than 1 s and cycles up to 40th order within 20 s on a single 1.8GHz CPU." While a CPU speed is given, a specific CPU model or manufacturer is not provided, and it's not explicitly stated that all main experiments were run on this hardware. |
| Software Dependencies | No | The paper mentions several software packages and libraries (e.g., scipy.integrate, scipy.fft, numpy.hanning, Py SINDy, torchdiffeq, Dynamical Systems.jl, Adam optimizer), but it does not specify version numbers for any of these dependencies, which is necessary for reproducibility. |
| Experiment Setup | Yes | To train the dend PLRNN in the VI framework, Adam (Kingma & Ba, 2015), with a batch size of 1000 and learning rate of 10 3 was used as the optimizer. For the training with BPTT, we used the Adam optimizer with an initial learning rate of 10 3 that was iteratively reduced during training down to 10 5. For each epoch we randomly sampled sequences of length Tseq = 500 ... from the total training data pool of each dataset, which are then fed into the reconstruction method in batches of size 16. Parameters A, W and h were initialized according to Talathi & Vartak (2016)... To find optimal hyper-parameters we performed a grid search within λ {0, 0.01, 0.1, 1, 10} (VI), τ {1, 5, 10, 25, 50, 100} (BPTT-TF), M {5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 75, 100}, and B {0, 1, 2, 5, 10, 20, 35, 50}. Hyper-parameters chosen for the benchmarks in Sec. 4 are reported in Table S1... |