Coupling Artificial Neurons in BERT and Biological Neurons in the Human Brain
Authors: Xu Liu, Mengyue Zhou, Gaosheng Shi, Yu Du, Lin Zhao, Zihao Wu, David Liu, Tianming Liu, Xintao Hu
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental results demonstrate 1) The activations of ANs and BNs are significantly synchronized; 2) the ANs carry meaningful linguistic/semantic information and anchor to their BN signatures; 3) the anchored BNs are interpretable in a neurolinguistic context. Overall, our study introduces a novel, general, and effective framework to link transformer-based NLP models and neural activities in response to language and may provide novel insights for future studies such as brain-inspired evaluation and development of NLP models. |
| Researcher Affiliation | Academia | Xu Liu1*, Mengyue Zhou1*, Gaosheng Shi1*, Yu Du1, Lin Zhao2, Zihao Wu2, David Liu3, Tianming Liu2, Xintao Hu1 1 School of Automation, Northwestern Polytechnical University 2 School of Computing, University of Georgia 3 Athens Academy {liu xu, zhou my, 2021202420, dddyyy}@mail.nwpu.edu.cn, {lin.zhao, zw63397}@uga.edu, {david.weizhong.liu, tianming.liu}@gmail.com, xhu@nwpu.edu.cn |
| Pseudocode | No | The paper describes its methods textually and with diagrams (e.g., Fig. 1, 2, 3), but it does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | No | The paper cites and links to several third-party tools used in the research (e.g., SPM, VS-DBN, Hugging Face's BERT, spaCy, Text-Attention-Heatmap Visualization tool) but does not provide a link to the authors' own source code implementation for the coupling framework described in the paper. |
| Open Datasets | Yes | We use the Narratives f MRI dataset (Nastase et al. 2021) in this study. |
| Dataset Splits | No | The paper describes data preprocessing and parameters for training the VS-DBN (e.g., hidden units, learning rate, batch size, epochs), but it does not explicitly mention train/validation/test splits for the primary task of coupling ANs and BNs or for the fMRI data beyond the VS-DBN training. |
| Hardware Specification | Yes | Model training is performed on a workstation with 10 Ge Force 1080Ti GPUs. |
| Software Dependencies | No | The paper mentions several software tools and libraries (e.g., SPM, VS-DBN, Hugging Face's BERT, spaCy, text attention heatmap visualization tool) but does not provide specific version numbers for these software components, which is necessary for reproducible software dependencies. |
| Experiment Setup | Yes | All the volumes in the two sessions are aggregated to train the VS-DBN with the following parameters: 512/256/128 hidden units in the 1st/2nd/3rd RBM layer, Gaussian (zeromean and a standard deviation of 0.01) initialization, learning rate 0.001/0.0005/0.0005, batch-size 20, L1 weightdecay rate 0.001/0.00005/0.00005, 100 training epochs, batch normalization. |