Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Target Propagation via Regularized Inversion for Recurrent Neural Networks
Authors: Vincent Roulet, Zaid Harchaoui
TMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We illustrate the algorithm with recurrent neural networks on long sequences in various sequence modeling problems and delineate the regimes in which the computational complexity of TP can be attractive compared to BP. ... Following the benchmark of Manchev & Spratling (2020), we observe that the proposed implementation of TP can perform better than classical gradient-based methods on several tasks involving RNNs. The code is made publicly available at https://github.com/vroulet/tpri. ... Section 4 Experiments |
| Researcher Affiliation | Academia | Vincent Roulet EMAIL Department of Statistics University of Washington Zaid Harchaoui EMAIL Department of Statistics University of Washington |
| Pseudocode | Yes | Algorithm 1 Stochastic learning with target propagation ... Algorithm 2 Proposed target propagation algorithm |
| Open Source Code | Yes | The code is made publicly available at https://github.com/vroulet/tpri. |
| Open Datasets | Yes | Data. We consider two synthetic datasets generated to present training difficulties for RNNs and several real datasets consisting of scanning images pixel by pixel to classify them (Hochreiter & Schmidhuber, 1997; Le et al., 2015; Manchev & Spratling, 2020). ... (i) grayscale handwritten digits given in the database MNIST (Le Cun & Cortes, 1998), (ii) colored objects from the database CIFAR10 (Krizhevsky, 2009) or (iii) grayscale images of clothes from the database Fashion MNIST (Xiao et al., 2017). ... Penn Treebank dataset (Marcinkiewicz, 1994) |
| Dataset Splits | No | The paper describes the datasets used (MNIST, CIFAR10, Fashion MNIST, Penn Treebank) and how they are processed (e.g., 'scanned pixel by pixel'), but does not provide specific training/test/validation dataset split percentages, absolute sample counts, or explicit references to predefined splits with access information. |
| Hardware Specification | Yes | All experiments were performed on GPUs using Nvidia Ge Force GTX 1080 Ti (12G memory). Each experiment only used one gpu at a time (clock speed 1.5 Ghz). |
| Software Dependencies | Yes | We used Python 3.8 and Py Torch 1.6. The RNN was coded using the cu DNN implementation available in Py Torch that is highly optimized for computing forward-backward passes on RNNs for gradient back-propagation. |
| Experiment Setup | Yes | Model. In both synthetic settings, we consider randomly generated mini-batches of size 20, a simple RNN with hidden states of dimension 100, and hyperbolic tangent activation. ... For the classification of images with sequential networks, we consider mini-batches of size 16 and a cross-entropy loss. ... We use a simple RNN architecture with hyperbolic tangent activations for the transitions of hidden states of size 256... Initialization and Hyper-Parameters. ... The learning rates of BP and the parameters of TP were found by a grid-search on a log10 basis and are presented in Table 1. |