Reviving and Improving Recurrent Back-Propagation
Authors: Renjie Liao, Yuwen Xiong, Ethan Fetaya, Lisa Zhang, KiJung Yoon, Xaq Pitkow, Raquel Urtasun, Richard Zemel
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | All experiments demonstrate that RBPs, especially the Neumann-RBP variant, are efficient and effective for optimizing convergent recurrent neural networks. |
| Researcher Affiliation | Collaboration | 1Department of Computer Science, University of Toronto 2Uber ATG Toronto 3Vector Institute 4Department of Electrical and Computer Engineering, Rice University 5Department of Neuroscience, Baylor College of Medicine 6Canadian Institute for Advanced Research. |
| Pseudocode | Yes | Algorithm 1 : Original RBP |
| Open Source Code | Yes | Our implementation based on PyTorch is publicly available1. 1https://github.com/lrjconan/RBP |
| Open Datasets | Yes | We train and test on 10 MNIST images. |
| Dataset Splits | Yes | The labeled documents are randomly split into 1%, 49% and 50% for training, validation and testing. |
| Hardware Specification | No | The paper mentions "GPU memory" but does not specify the exact GPU models, CPU types, or other detailed hardware specifications used for running the experiments. |
| Software Dependencies | No | The paper states "Our implementation based on PyTorch is publicly available" but does not specify the version number of PyTorch or other software dependencies. |
| Experiment Setup | Yes | using an initial learning rate of exp( -1) and momentum 0.5. The optimization is on a fully connected network with 4 layers, of sizes 784, 50, 50, and 50... We use tanh non-linearities and train on 10, 000 examples on MNIST... We use Adam (Kingma & Ba, 2014) as the meta optimizer and set the learning rate to 0.05. |