Joint processing of linguistic properties in brains and language models
Authors: SUBBAREDDY OOTA, Manish Gupta, Mariya Toneva
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We investigate this correspondence via a direct approach, in which we eliminate information related to specific linguistic properties in the language model representations and observe how this intervention affects the alignment with f MRI brain recordings obtained while participants listened to a story. |
| Researcher Affiliation | Collaboration | Subba Reddy Oota1,2, Manish Gupta3, Mariya Toneva2 1Inria Bordeaux, France, 2MPI for Software Systems, Saarbrücken, Germany, 3Microsoft, India |
| Pseudocode | No | The paper includes a schematic diagram (Figure 1) to illustrate its approach but does not contain any sections or figures explicitly labeled "Pseudocode" or "Algorithm", nor structured steps formatted like code. |
| Open Source Code | Yes | We make the code publicly available1. 1https://github.com/subbareddy248/lingprop-brain-alignment |
| Open Datasets | Yes | We use a dataset of f MRI recordings that are openly available (Nastase et al., 2021) and correspond to 18 participants listening to a natural story. ... We analyze the Narratives-21st year dataset (Nastase et al., 2021), which is one of the largest publicly available f MRI datasets (in terms of number of samples per participant). |
| Dataset Splits | Yes | Cross-Validation The ridge regression parameters were fit using 4-fold cross-validation. All the data samples from K-1 folds were used for training, and the generalization was tested on samples from the left-out fold. ... the best λ was chosen by tuning on validation data |
| Hardware Specification | Yes | All experiments were conducted on a machine with 1 NVIDIA GEFORCE-GTX GPU with 16GB GPU RAM. |
| Software Dependencies | No | The paper mentions using specific software components like Hugging Face (for BERT and GPT-2) and Stanford CoreNLP stanza library, but it does not provide specific version numbers for these software dependencies, only stating general names. |
| Experiment Setup | Yes | The ridge regression parameters were fit using 4-fold cross-validation. All the data samples from K-1 folds were used for training, and the generalization was tested on samples from the left-out fold. ... We used banded ridge-regression with the following parameters: MSE loss function, and L2-decay (λ) varied from 101 to 103; the best λ was chosen by tuning on validation data; the number of cross-validation runs was 4. |