Inferring stochastic low-rank recurrent neural networks from neural data
Authors: Matthijs Pals, A Erdem Sağtekin, Felix Pei, Manuel Gloeckler, Jakob H Macke
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our method on several datasets consisting of both continuous and spiking neural data, where we obtain lower dimensional latent dynamics than current state of the art methods. Additionally, for low-rank models with piecewise-linear nonlinearities, we show how to efficiently identify all fixed points in polynomial rather than exponential cost in the number of units, making analysis of the inferred dynamics tractable for large RNNs. Our method both elucidates the dynamical systems underlying experimental recordings and provides a generative model whose trajectories match observed variability. |
| Researcher Affiliation | Academia | 1Machine Learning in Science, Excellence Cluster Machine Learning, University of Tübingen, Germany 2Tübingen AI Center, Tübingen, Germany 3Graduate Training Centre of Neuroscience, University of Tübingen, Germany 4Department Empirical Inference, Max Planck Institute for Intelligent Systems, Tübingen, Germany |
| Pseudocode | Yes | Algorithm 1: Improved exhaustive search for all fixed points |
| Open Source Code | Yes | Code to reproduce our results is available at https://github.com/mackelab/smc_rnns. |
| Open Datasets | Yes | We used openly accessible electroencephalogram (EEG) data from [42, 43] ( https://www.physionet.org/content/eegmmidb/1.0.0/, ODC-BY licence). We used openly accessible neurophysiological data recorded from layer CA1 of the right dorsal hippocampus [47, 48] (https://crcns.org/data-sets/hc/hc-2/about-hc-2. We used openly accessible neurophysiological data recorded from hippocampal CA1 region [50 52] (https://crcns.org/data-sets/hc/hc-11/about-hc-11). We used the publicly available MC_Maze dataset from the Neural Latents Benchmark (NLB) [56] (https://dandiarchive.org/dandiset/000128, CC-BY-4.0 licence). |
| Dataset Splits | Yes | We used the first 80 percent of the data for training, and the rest was saved for testing purposes. After training, we selected the model with the best co-smoothing score on the validation split and submitted its predictions to the benchmark for the final evaluation. |
| Hardware Specification | Yes | We used a workstation with a NVIDIA Ge Force RTX 3090 GPU for these runs. Models were trained using NVIDIA RTX 2080 TI GPUs on a compute cluster. |
| Software Dependencies | No | During training we minimise the variational SMC ELBO [33 35] (Eq.7) with stochastic gradient descent, using the RAdam [74] optimiser in Pytorch [72]. While Pytorch is mentioned, a specific version number is not provided in the paper's experimental setup details. |
| Experiment Setup | Yes | Our models are (unless noted otherwise) initialized as follows: Hij U[ 1 Ninp , 1 Ninp ], b 0, a .9, Σz .01I, Σz1 I, µz1 0, where W and b are the output weights and biases respectively. For Gaussian observations we initialise Σy .01I. For all three experiments, we used k = 64 particles, batch-sizes of 10, and decreased the learning rate exponentially from 10 3 to 10 5. For Fig. 3a we trained for 1000 epochs of 200 trials, for Fig. 3b for 1500 epochs of 400 trials and for Fig. 3c for 200 epochs of 800 trials. |