Implicit Bias of Linear RNNs
Authors: Melikasadat Emami, Mojtaba Sahraee-Ardakan, Parthe Pandit, Sundeep Rangan, Alyson K Fletcher
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The theory is validated with both synthetic and real data experiments. |
| Researcher Affiliation | Academia | 1Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, USA 2Department of Statistics, University of California, Los Angeles, Los Angeles, USA 3Department of Electrical and Computer Engineering, New York University, Brooklyn, New York, USA. Correspondence to: Melikasadat Emami <emami@ucla.edu>. |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain an explicit statement offering open-source code or a link to a code repository. |
| Open Datasets | Yes | We validate our theoretical results on a number of synthetic and real data experiments... We also validated our theory using spikes rate data from the macaque primary somatosensory cortex (S1) (Benjamin et al., 2018). |
| Dataset Splits | No | The paper mentions '50 training sequences and 50 test sequences' but does not specify a separate validation split or dataset. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies used in the experiments. |
| Experiment Setup | Yes | Full batch gradient descent is used with lr = 10 4. ... For both cases we used mini-batch (batch size = 128) gradient descent with lr = 10 4. ... the linear RNN has n = 1000 hidden states and the sequence length T = 15. Also, νW = 0.3 and νF = νC = 1. |