A Statistical Investigation of Long Memory in Language and Music
Authors: Alexander Greaves-Tunnell, Zaid Harchaoui
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We report experimental results obtained on a wideranging collection of music and language data, confirming the (often strong) long-range dependencies that are observed by practitioners. However, we show evidence that this property is not adequately captured by a variety of RNNs trained to benchmark performance on a language dataset.1 |
| Researcher Affiliation | Academia | 1Department of Statistics, University of Washington, Seattle, USA. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | 1Code corresponding to these experiments, including an illustrative Jupyter notebook, is available for download at https: //github.com/alecgt/RNN_long_memory. |
| Open Datasets | Yes | We evaluate long memory in three different sources of English language text data: the Penn Tree Bank training corpus (Marcus et al., 1993), the training set of the Children s Book Test from Facebook s b Ab I tasks (Weston et al., 2016), and the King James Bible. |
| Dataset Splits | No | The paper mentions training and test sets but does not explicitly describe validation dataset splits (e.g., percentages, sample counts, or specific files/citations for validation data). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, processor types, memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions using "GloVe embeddings" and refers to "deep recurrent neural networks" but does not specify software dependencies with version numbers (e.g., PyTorch, TensorFlow, Python versions). |
| Experiment Setup | No | The paper states that the "architecture is identical to the small LSTM model in (Zaremba et al., 2014)", implying inherited setup details, but it does not explicitly list concrete hyperparameter values, training configurations, or system-level settings within the main text of this paper. |