Complex Gated Recurrent Neural Networks
Authors: Moritz Wolter, Angela Yao
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We test our cg RNN on two benchmark synthetic tasks: the memory problem and the adding problem [12]. These problems are designed especially to challenge RNNs, and require the networks to store information over time scales on the order of hundreds of time steps. |
| Researcher Affiliation | Academia | Moritz Wolter Institute for Computer Science University of Bonn wolter@cs.uni-bonn.de Angela Yao School of Computing National University of Singapore yaoa@comp.nus.edu.sg |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Source code available at https://github.com/v0lta/Complex-gated-recurrent-neural-networks |
| Open Datasets | Yes | We follow the same experimental setting as [28], working with the full Human 3.6M dataset [14]. ... We use the Music-Net dataset [35], which consists of 330 classical music recordings... |
| Dataset Splits | Yes | For training, we use six of the seven actors and test on actor five. ... We use the Music-Net dataset [35], which consists of 330 classical music recordings, of which 327 are used for training and 3 are held out for testing. |
| Hardware Specification | Yes | We also gratefully acknowledge NVIDIA s donation of a Titan X Pascal GPU. |
| Software Dependencies | No | The paper states 'We work in Tensorflow' but does not provide specific version numbers for TensorFlow or any other software dependencies. |
| Experiment Setup | Yes | We work in Tensorflow, using RMS-prop to update standard weights and the multiplicative Stiefelmanifold update as described Equation 8 for all unitary state transition matrices. The unitary state transition matrices are initialized the same as [1] as the product of component unitary matrices. All other weights are initialized using the uniform initialisation method recommended in [8], i.e. U[ l, l] with l = p 6/(nin + nout), where nin and nout are the input and output dimensions of the tensor to be initialised. All biases are intialized as zero, with the exception of the gate biases br and bz, which are initialized at 4 to ensure fully open gates and linear behaviour at the start of training. All synthetic tasks are run for 2 104 iterations with a batch-size of 50 and a constant learning rate of 0.001 for both the RMS-Prop and the Stiefel-Manifold updates. For the human motion prediction task, we adopt the state-of-the-art implementation of [28]... The learning rate and batch size are kept the same (0.005, 16) though we reduce our state size to 512... For music transcription... All cells are set with nh = 1024; the learning rate is set to 0.0001 and batch size to 5. |