Initialization matters: Orthogonal Predictive State Recurrent Neural Networks
Authors: Krzysztof Choromanski, Carlton Downey, Byron Boots
ICLR 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We compare the performance of OPSRNNs with that of LSTMs as well as conventional PSRNNs on a number of robotics tasks, and show that OPSRRNs are consistently superior on all tasks. Exhaustive experiments conducted on several robotics task confirm our theoretical findings. |
| Researcher Affiliation | Collaboration | Krzysztof Choromanski Google Brain kchoro@google.com Carlton Downey Carnegie Mellon University cmdowney@cs.cmu.edu Byron Boots Georgia Tech bboots@cc.gatech.edu |
| Pseudocode | No | The paper provides mathematical descriptions and textual explanations of methods, but does not include any formally structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about making the source code available or provide a link to a code repository. |
| Open Datasets | Yes | Swimmer We consider the 3-link simulated swimmer robot from the open-source package Open AI gym.2. Handwriting This is a digital database available on the UCI repository (Alpaydin & Alimoglu, 1998) Moving MNIST Pairs of MNIST digits bouncing around inside of a box according to ideal physics. http://www.cs.toronto.edu/ nitish/unsupervised_video/. |
| Dataset Splits | Yes | We collect 25 trajectories from a robot that is trained to swim forward (via the cross entropy with a linear policy), with a train/test split of 20/5. We use a train/test split of 40/8. We use 25 trajectories with a train/test split of 20/5. We use 1000 randomly selected videos, split evenly between train and test. |
| Hardware Specification | No | The paper mentions software used (Tensorflow framework in Python) but does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory) used for the experiments. |
| Software Dependencies | No | All models were implemented using the Tensorflow framework in Python. (No version numbers provided). |
| Experiment Setup | Yes | In two-stage regression we use history (similarly future) features consisting of the past (next) 2 observations concatenated together. We use a ridge-regression parameter of 10( 2) (this is consistent with the values suggested in Boots et al. (2013); Downey et al. (2017)). The kernel width is set to the median pairwise (Euclidean) distance between neighboring data points. We use a fixed learning rate of 0.1 for BPTT with a BPTT horizon of 20. We use a single layer PSRNN. |