Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Spell Once, Summon Anywhere: A Two-Level Open-Vocabulary Language Model
Authors: Sebastian J. Mielke, Jason Eisner6843-6850
AAAI 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comparing to baselines (including a novel strong baseline), we beat previous work and establish state-of-the-art results on multiple datasets. |
| Researcher Affiliation | Academia | Sebastian J. Mielke, Jason Eisner Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA EMAIL, EMAIL |
| Pseudocode | No | The paper describes the model mathematically and textually but does not contain structured pseudocode or algorithm blocks in the main body. |
| Open Source Code | Yes | Code at github.com/sjmielke/spell-once. |
| Open Datasets | Yes | We evaluate on two open-vocabulary datasets, Wiki Text-2 (Merity et al. 2017) and the Multilingual Wikipedia Corpus (Kawakami, Dyer, and Blunsom 2017). |
| Dataset Splits | Yes | Bits per character (lower is better) on the dev and test set of Wiki Text-2 for our model and baselines... |
| Hardware Specification | No | The paper mentions running experiments on 'GPUs' and 'computational resources at the Maryland Advanced Research Computing Center (MARCC)' but does not provide specific hardware details like GPU models, CPU types, or memory amounts. |
| Software Dependencies | No | The paper mentions using PyTorch and AWD-LSTM but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | No | The paper states that 'A detailed discussion of all hyperparameters can be found in Appendix B,' but does not include specific hyperparameter values or detailed training configurations within the main text. |