BrainLM: A foundation model for brain activity recordings
Authors: Josue Ortega Caro, Antonio Henrique de Oliveira Fonseca, Syed A Rizvi, Matteo Rosati, Christopher Averill, James L Cross, Prateek Mittal, Emanuele Zappala, Rahul Madhav Dhodapkar, Chadi Abdallah, David van Dijk
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We introduce the Brain Language Model (Brain LM), a foundation model for brain activity dynamics trained on 6,700 hours of f MRI recordings. Utilizing self-supervised masked-prediction training, Brain LM demonstrates proficiency in both fine-tuning and zero-shot inference tasks. Fine-tuning allows for the accurate prediction of clinical variables like age, anxiety, and PTSD as well as forecasting of future brain states. Critically, the model generalizes well to entirely new external cohorts not seen during training. |
| Researcher Affiliation | Academia | 1Wu Tsai Institute, 2Interdepartmental Neuroscience Program, Yale University, 3Baylor College of Medicine, 4 Department of Computer Science, 5 Yale School of Medicine, 7 University of Southern California, 8 Interdepartmental Program in Computational Biology & Bioinformatics, 9 Internal Medicine, 10 Cardiovascular Research Center, Yale University,11 Department of Mathematics and Statistics, Idaho State University |
| Pseudocode | No | The paper includes Figure 2 which visually depicts the 'Brain LM architecture and training procedure', but it is a diagram and not structured pseudocode or an algorithm block. |
| Open Source Code | No | The paper states 'We will make all pretrained models available via Hugging Face.' in Section 4.1. However, it does not explicitly provide a link or state that the source code for the methodology itself will be released. |
| Open Datasets | Yes | We leveraged two large-scale publicly available datasets the UK Biobank (UKB) (Miller et al., 2016) and the Human Connectome Project (HCP) (Elam et al., 2021). |
| Dataset Splits | No | The paper states: 'Our model was trained on 80% of the UKB dataset (61,038 recordings) and evaluated on the held-out 20% and the full HCP dataset.' This clearly defines the training and test splits but does not mention a separate validation split for hyperparameter tuning. |
| Hardware Specification | No | The paper mentions details about the fMRI data acquisition hardware ('Recordings were acquired on a Siemens 3T scanner at 0.735s temporal resolution'), but does not provide any specific hardware details (e.g., CPU, GPU models, memory, or cluster specifications) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions or library versions). It only mentions the optimization algorithm used: 'optimized via the Adam algorithm'. |
| Experiment Setup | Yes | During training, we selected random subsequences spanning 200 timesteps from each f MRI recording. These parcel time series were then dissected into blocks of 20 timesteps, leading to 10 non-overlapping segments per subsequence. These segments were transformed into 512-dimensional vectors, and a masking operation was performed at rates of 20%, 75%, or 90%. ... The training regimen involved batches of 512 samples, optimized via the Adam algorithm across a span of 100 epochs. The optimization goal was to minimize the mean squared error between the original and the reconstructed signals (visualized in Figure 2). ... To adapt Brain LM for predicting clinical variables from f MRI recordings, we augmented the pretrained encoder with a 3-layer MLP head. ... To mitigate overfitting during the fine-tuning process, we introduced a 10% dropout to the activations of both the Brain LM encoder and its MLP head. |