Interpretable multi-timescale models for predicting fMRI responses to continuous natural speech
Authors: Shailee Jain, Vy Vo, Shivangi Mahto, Amanda LeBel, Javier S. Turek, Alexander Huth
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work we construct interpretable multi-timescale representations by forcing individual units in an LSTM LM to integrate information over specific temporal scales. This allows us to explicitly and directly map the timescale of information encoded by each individual f MRI voxel. Further, the standard f MRI encoding procedure does not account for varying temporal properties in the encoding features. We modify the procedure so that it can capture both shortand long-timescale information. This approach outperforms other encoding models, particularly for voxels that represent long-timescale information. |
| Researcher Affiliation | Collaboration | Shailee Jain Department of Computer Science The University of Texas at Austin Austin, TX 78712 shailee@cs.utexas.edu; Vy A. Vo Brain-Inspired Computing Lab Intel Labs Hillsboro, OR 97124 vy.vo@intel.com |
| Pseudocode | No | The paper describes the models and methods in detail but does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain an explicit statement about the release of its source code or a link to a code repository. |
| Open Datasets | Yes | To build encoding models for language, we used data from an f MRI experiment comprising 6 human subjects (3 female) listening to spoken narrative stories from The Moth Radio Hour (an English language podcast) [13]. ... The multi-timescale LSTM LM was pre-trained on Wiki Text-2 [23] - [13] https://themoth.org. The moth radio hour, 2020. - [23] refers to Merity et al. 2017, which is a common citation for Wiki Text-2. |
| Dataset Splits | Yes | To find the best regularization coefficient for each voxel, the regression procedure was repeated 50 times, each time holding out a random sample of 5000 TRs (125 blocks of 40 consecutive TRs) from the training set to use for validation. |
| Hardware Specification | No | The paper mentions 'HPC resources provided by the Texas Advanced Computing Center (TACC) at The University of Texas at Austin' and 'GPUs donated by NVIDIA' but does not specify exact GPU models, CPU models, or other detailed hardware specifications. |
| Software Dependencies | No | The paper describes the LSTM architecture and refers to previous work [22] for training details and hyperparameters but does not explicitly list specific software dependencies with their version numbers (e.g., 'PyTorch 1.x', 'Python 3.x'). |
| Experiment Setup | Yes | The LSTM architecture is adopted from [22]. It has 3 layers, with 1150 hidden state units in the first two layers, and 400 units in the third. ... Timescales in the second layer are distributed according to an inverse gamma distribution with a shape parameter α = 0.56 and scale parameter β = 1 [11]. ... The multi-timescale LSTM LM was pre-trained on Wiki Text-2 [23]... Further details on pre-training and fine-tuning can be found in Supplementary Section 2, including hyper-parameters that were modified from [22]. |