Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Jacobian-Based Interpretation of Nonlinear Neural Encoding Model
Authors: Xiaohui Gao, Haoran Yang, cheng yue, Mengfei Zuo, Yiheng Liu, Peiyang Li
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Centered on proposing JNE as a novel interpretability metric, we validated its effectiveness through controlled simulation experiments on various activation functions and network architectures, and further verified it on real f MRI data, demonstrating a hierarchical progression of nonlinear characteristics from primary to higher-order visual cortices, consistent with established cortical organization. |
| Researcher Affiliation | Academia | 1Northwestern Polytechnical University 2Beijing Jiaotong University 3Chongqing University of Posts and Telecommunications |
| Pseudocode | No | To address the above issues, we propose a novel interpretability metric for nonlinearity Jacobianbased Nonlinearity Evaluation (JNE) designed to quantify the nonlinear characteristics of neural encoding models, thereby approximating voxel-level BOLD nonlinearity. The core idea is to calculate the local linear mappings (i.e., local derivatives, also known as Jacobian matrices) of the neural encoding model under different input stimuli, and to evaluate the level of nonlinearity by statistically measuring their variability across inputs (theoretical description in Section A.1). Intuitively, an ideal linear neural encoding model should maintain a constant mapping across different inputs, resulting in a JNE value of 0; in contrast, if the model exhibits significant nonlinearity, the Jacobian matrices will vary across inputs, yielding a higher JNE value. Centering on JNE, we validate via simulations for nonlinearity quantification, then apply to f MRI for approximating visual cortex BOLD nonlinearity. JNE serves as a conditional interpretability metric for approximating voxel-level BOLD nonlinearity, rather than a direct neural measure, due to BOLD confounds. In summary, the main contributions of this study include: We propose JNE as a novel interpretability metric for nonlinear neural encoding models, which quantifies nonlinearity by statistically measuring the dispersion of input-output Jacobian matrices (Section 2.4), and theoretical derivation indicates that JNE can serve as an approximate measure of BOLD response nonlinearity (Section A.8). Controlled simulation experiments validate the effectiveness of JNE in quantifying the nonlinearity of output responses (Sections 3.1 and A.7); Our results demonstrate that inferring nonlinear properties by comparing R2 values between linear and nonlinear neural encoding models is insufficient to reveal the brain s nonlinear responses, especially when deep visual representations already embed nonlinear transformations (Section 3.2); We find that the primary visual cortex tends to exhibit more linear responses, while higher-order visual cortices show stronger nonlinear characteristics under natural visual stimulation (Section 3.3). Overall, a hierarchical structure emerges with increasing nonlinearity from the primary to intermediate to higher-order visual cortex (Section 3.4); We further define JNE-SS (Section A.10), which characterizes the nonlinear properties of individual samples at specific voxels. Our results indicate that BOLD response nonlinearity exhibits sample-selective preferences (Section 3.5). |
| Open Source Code | Yes | Code available at https://github.com/Gaitxh/JNE. |
| Open Datasets | Yes | In this study, we used the Natural Scenes Dataset (NSD) [30], which provides whole-brain 7T f MRI responses from multiple subjects while viewing natural scene images. We focused on four subjects (S1, S2, S5, S7) who completed the full experimental protocol as the primary participants in our analysis. Each image and its corresponding averaged beta map were treated as a single sample. The dataset was split into training, validation, and testing sets with a ratio of 8:1:1 (8000:1000:1000 samples), ensuring no subject overlap between the training and validation sets, while the test set included repeated samples across subjects. We employed the pre-trained CLIP-Vi T [31]1 image encoder to obtain computational representations of the visual stimuli (Fig. 2a). Model performance was evaluated using R2, and statistical significance was assessed through bootstrap resampling and FDR correction to identify statistically significant activated voxels (see Section A.2 for details). |
| Dataset Splits | Yes | The dataset was split into training, validation, and testing sets with a ratio of 8:1:1 (8000:1000:1000 samples), ensuring no subject overlap between the training and validation sets, while the test set included repeated samples across subjects. |
| Hardware Specification | Yes | approximately several days (Intel Xeon E5-2620 v4 CPU in this study). We optimized this process through explicit forward-mode analytical differentiation. For the nonlinear model employed in our study, the forward structure is as follows, where h(k) denotes the hidden activations at layer k, bk represents the bias vectors, WF Ck are the weight matrices, and ϕ1, ϕ2, ϕ3 are the activation functions: |
| Software Dependencies | No | The mean squared error (MSE) [51] is used as the loss function in the encoder, and the training process of the encoder utilizes the Adam optimizer [52] for parameter optimization. To prevent overfitting, early stopping is employed, ceasing training if the validation loss fails to improve over 8 consecutive epochs. Additionally, the model parameters that demonstrate the optimal performance on the validation set are recorded and preserved. |
| Experiment Setup | Yes | The mean squared error (MSE) [51] is used as the loss function in the encoder, and the training process of the encoder utilizes the Adam optimizer [52] for parameter optimization. To prevent overfitting, early stopping is employed, ceasing training if the validation loss fails to improve over 8 consecutive epochs. Additionally, the model parameters that demonstrate the optimal performance on the validation set are recorded and preserved. To further assess JNE s practical utility in multilayer network structures, we developed an ANN simulation framework (Fig.3c). The model consisted of a three-layer fully connected residual architecture and was not subject to any training it was used solely for forward propagation to obtain the Jacobian matrices between inputs and outputs. Nonlinearity was introduced via two plug-in locations (Loc1 and Loc2), where activation functions could be flexibly inserted in a fully controlled manner (Fig.3c and d). This setup allowed us to systematically evaluate JNE s sensitivity to different nonlinear configurations without altering the overall network architecture. To simulate the real neural encoding models used in this study, we set the input dimension to 768 (mimicking CLIP-Vi T image representations) and generated an input matrix of size 1000 768, with the corresponding output being a simulated response vector of size 1000 1. For each input sample, we computed the Jacobian of the output with respect to the input and evaluated the overall nonlinearity using the standard JNE pipeline. We conducted comparative analyses for four activation functions under different Loc1/Loc2 configurations (none/shallow/deep/both layers). Each configuration was sampled 200 times to ensure stable statistical estimates (see Section A.6 for details). |