Inverse Approximation Theory for Nonlinear Recurrent Neural Networks
Authors: Shida Wang, Zhong Li, Qianxiao Li
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our theoretical results are confirmed by numerical experiments. |
| Researcher Affiliation | Collaboration | Shida Wang Department of Mathematics National University of Singapore e0622338@u.nus.edu Zhong Li Microsoft Research Asia lzhong@microsoft.com Qianxiao Li Department of Mathematics Institute for Functional Intelligent Materials National University of Singapore qianxiao@nus.edu.sg |
| Pseudocode | No | The paper provides mathematical derivations and equations but no pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code are attached in the supplementary materials. |
| Open Datasets | Yes | We train the nonlinear RNNs on the MNIST dataset for 10 epochs. The batch size is 128 while the train set size and test set size are 12800. |
| Dataset Splits | No | The stopping criterion is the validation loss achieving 10^-8. The paper mentions 'validation loss' and 'validation accuracy' but does not specify the size or percentage of the validation set or how it was split from the main dataset. |
| Hardware Specification | No | The paper does not specify any hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions 'Adam' as the optimizer but does not specify software versions for libraries (e.g., PyTorch, TensorFlow) or the Python interpreter. |
| Experiment Setup | Yes | In the experiments to approximate the nonlinear functionals by nonlinear RNNs, we train each model for 1000 epochs, the stopping criterion is the validation loss achieving 10^-8. The optimizer used is Adam with initial learning rate 0.005. The loss function is mean squared error. The batch size is 128 while the train set size and test set size are 12800. |