Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets
Authors: Edo Cohen-Karlik, Itamar Menuhin-Gruman, Raja Giryes, Nadav Cohen, Amir Globerson
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments corroborate our theory, demonstrating extrapolation via learning low-dimensional state spaces with both linear and non-linear RNNs. ... In this section we present experiments corroborating our theoretical analysis (Section 4). |
| Researcher Affiliation | Collaboration | Edo Cohen-Karlik1, Itamar Menuhin-Gruman1, Raja Giryes1, Nadav Cohen1 & Amir Globerson1,2 1Tel Aviv University 2Google Research |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | No statement explicitly providing concrete access to source code for the methodology described in this paper was found. |
| Open Datasets | No | The paper mentions 'input sequences are drawn from a whitened distribution' and 'data is sampled from a Gaussian with zero mean and scale of 1' (Appendix C.1), indicating synthetic data generation rather than the use of a named, publicly available dataset with access information. |
| Dataset Splits | No | The paper does not provide specific dataset split information (e.g., exact percentages or sample counts for training, validation, and test sets). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper states 'All the experiments are implemented using Py Torch' (Appendix C.1), but it does not specify any version numbers for PyTorch or other software dependencies. |
| Experiment Setup | Yes | We use 15K optimization steps with Adam optimizer and a learning rate of 10 3. ... The batch size is set to 100... The examination of the effect of initialization scale presented in Section B.1.2 is done with learning rate scheduler torch.optim.lr_scheduler.Multi Step LR using milestones at r5000, 10000, 15000, 30000s and a decaying factor of γ 0.1. |