Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Understanding Recurrent Neural Networks Using Nonequilibrium Response Theory
Authors: Soon Hoe Lim
JMLR 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | The present work is a contribution towards a deeper understanding of how RNNs process input signals using the response theory from nonequilibrium statistical mechanics. For a class of continuous-time stochastic RNNs (SRNNs) driven by an input signal, we derive a Volterra type series representation for their output. This representation is interpretable and disentangles the input signal from the SRNN architecture. The kernels of the series are certain recursively defined correlation functions with respect to the unperturbed dynamics that completely determine the output. Exploiting connections of this representation and its implications to rough paths theory, we identify a universal feature the response feature, which turns out to be the signature of tensor product of the input signal and a natural support basis. In particular, we show that SRNNs, with only the weights in the readout layer optimized and the weights in the hidden layer kept fixed and not optimized, can be viewed as kernel machines operating on a reproducing kernel Hilbert space associated with the response feature. |
| Researcher Affiliation | Academia | Soon Hoe Lim EMAIL Nordita KTH Royal Institute of Technology and Stockholm University Stockholm 106 91, Sweden |
| Pseudocode | No | The paper primarily presents mathematical derivations and theoretical analysis. There are no sections, figures, or text blocks explicitly labeled as 'Pseudocode' or 'Algorithm', nor are any structured, step-by-step procedures provided in a code-like format. |
| Open Source Code | No | The paper focuses on theoretical contributions and mathematical derivations. It does not contain any statements about making source code available, nor does it provide links to code repositories or supplementary materials for code implementation. |
| Open Datasets | No | The paper is theoretical in nature, presenting mathematical models and analysis of recurrent neural networks. It does not perform any empirical experiments or use specific datasets, thus no information about publicly available datasets is provided. |
| Dataset Splits | No | The paper is a theoretical work focusing on mathematical derivations and does not involve experimental evaluation using datasets. Therefore, no information regarding training, validation, or test dataset splits is provided. |
| Hardware Specification | No | The paper is theoretical, focused on mathematical analysis and derivations, and does not describe any experiments that would require computational resources. Therefore, no specific hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical, presenting mathematical analysis rather than empirical experimentation. As such, it does not describe any software implementation, and therefore, no specific software dependencies or version numbers are mentioned. |
| Experiment Setup | No | The paper is entirely theoretical, presenting mathematical models and derivations without any empirical experiments. Therefore, there are no experimental setup details, hyperparameters, or training configurations described. |