Neuroformer: Multimodal and Multitask Generative Pretraining for Brain Data
Authors: Antonis Antoniades, Yiyi Yu, Joe S Canzano, William Yang Wang, Spencer Smith
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We first trained Neuroformer on simulated datasets, and found that it both accurately predicted simulated neuronal circuit activity, and also intrinsically inferred the underlying neural circuit connectivity, including direction. We used an ablation study to show that joint training on neuronal responses and behavior boosted performance, highlighting the model s ability to associate behavioral and neural representations in an unsupervised manner. |
| Researcher Affiliation | Academia | Antonis Antoniades , Yiyi Yu, Joseph Canzano, William Wang, Spencer La Vere Smith University of California, Santa Barbara |
| Pseudocode | No | No pseudocode or algorithm blocks were found. |
| Open Source Code | Yes | Code: https://github.com/a-antoniades/Neuroformer |
| Open Datasets | No | The paper states: 'We simulated a spiking neural network characterized by directional connectivity with three hub neurons using Brian2 simulator (Stimberg et al., 2019) (Hub-net simulation).' and 'These consisted of neuronal activity recorded from awake mice, which were either passively viewing visual stimuli (Passive-Stim data), or actively engaged in a visually-guided navigation task (Visnav data).' No concrete access information (link, DOI, repository name, or citation to a public version) for the datasets used was provided. |
| Dataset Splits | No | The paper states: 'We trained all Neuroformer models with 8 layers and 8 heads for the decoder and each of the cross-attention modules, with an embedding dimension of 256 and an 80/20 train/test split.' It explicitly mentions a train/test split but does not specify a separate validation split or percentages for it. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory) used for running experiments were provided. |
| Software Dependencies | No | The paper mentions 'Py Torch' and 'Brian2 simulator (Stimberg et al., 2019)' but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | We trained all Neuroformer models with 8 layers and 8 heads for the decoder and each of the cross-attention modules, with an embedding dimension of 256 and an 80/20 train/test split... We used Adam W optimizer (Loshchilov & Hutter, 2019) in Py Torch. Learning rate was 2e-4, with linear warmup and cosyne decay schedules. Batch size was kept at 224 for pretraining and 32 for fine-tuning. |