Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Revisiting Bi-Linear State Transitions in Recurrent Neural Networks
Authors: Reza Ebrahimi, Roland Memisevic
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we revisit bilinear operations, which involve multiplicative interactions between hidden units and input embeddings. We demonstrate theoretically and empirically that they constitute a natural inductive bias for representing the evolution of hidden states in state tracking tasks. |
| Researcher Affiliation | Industry | M.Reza Ebrahimi Qualcomm AI Research EMAIL Roland Memisevic Qualcomm AI Research EMAIL |
| Pseudocode | No | The paper includes mathematical equations and descriptions of models but does not contain a clearly labeled 'Pseudocode' or 'Algorithm' block, nor structured steps formatted like code. |
| Open Source Code | No | Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: While code release is not currently planned, all models, training algorithms and data generation are described in great detail in the supplemental material for easy reproducibility. All experiments involve very simple existing models run on auto-generated standard datasets. |
| Open Datasets | No | Tasks: To evaluate the state-tracking capabilities of the bilinear RNN model variants introduced previously, we use the following three tasks: modular addition, random state machine, and modular arithmetic. In the modular addition task, the model processes a sequence of integers, each randomly drawn from the set Zm = {0, , m 1}, and is required to predict their sum modulo m. For the random state machine task, the model must simulate a randomly generated finite-state machine... Finally, the modular arithmetic task involves processing a sequence alternating between integers from Zm and arithmetic operators (from the set {+, , }); the model must compute and output the result of these operations... For all tasks, multi-digit integers are tokenized into single tokens. We refer the reader to Appendix B.1 for examples and additional details on each task. ... For all tasks, to generate a training sample, we first randomly select the number of inputs n U(2, N), where N is the maximum training sequence length. We then select n input symbols from {0, 1, . . . , m 1} uniformly at random with replacement. |
| Dataset Splits | Yes | We trained the models described above on the three tasks, on instances of lengths 2-10, and evaluated on instances of length 500. ... Validation Accuracy (Length 2-10) OOD Accuracy (Length 500) |
| Hardware Specification | Yes | All experiments were conducted on a cluster of A100 GPU nodes. |
| Software Dependencies | No | The paper mentions software components like 'ADAM optimizer', 'GPT-2 architecture', and 'Mamba-1', but it does not provide specific version numbers for these or any other software libraries (e.g., Python, PyTorch, TensorFlow) used in the implementation. |
| Experiment Setup | Yes | All models were trained using the ADAM optimizer with three learning rates (10 3, 10 4, 10 5), and the configuration yielding the best performance was selected for reporting. All models were trained from random initializations, without learning rate scheduling, weight decay, or dropout. In addition, the parameters of bilinear models (and variants) were initialized from a uniform distribution U( 0.01, 0.01). Training was conducted for 100,000 steps with a batch size of 64. An early stopping criterion was applied if the validation loss fell below 10 5. For these experiments, training examples were randomly sampled at each training step with input sequence lengths ranging from 2 to 10, while models were evaluated on inputs of length 500. ... A consistent hidden and input dimensionality of 256 is used across all models. |