Latent Normalizing Flows for Discrete Sequences
Authors: Zachary Ziegler, Alexander Rush
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments consider common discrete sequence tasks of character-level language modeling and polyphonic music generation. Our results indicate that an autoregressive flow-based model can match the performance of a comparable autoregressive baseline, and a non-autoregressive flow-based model can improve generation speed with a penalty to performance. |
| Researcher Affiliation | Academia | 1School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA. |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at https://github.com/harvardnlp/TextFlow. |
| Open Datasets | Yes | We use the Penn Treebank dataset, with the standard preprocessing as in (Mikolov et al., 2012). The dataset consists of approximately 5M characters, with rare words replaced by <unk> and a character-level vocabulary size of V = 51. Next we consider the polyphonic music modeling task (Boulanger-Lewandowski et al., 2012). |
| Dataset Splits | No | The paper mentions evaluating on a 'held-out test set' and refers to 'validation' in a figure caption, and describes dataset splitting into sentences, but it does not provide specific numerical percentages or counts for train/validation/test splits in the main text. |
| Hardware Specification | Yes | Experiments are run on a single Tesla V100 GPU with a batch size of one, with the IAF / SCF model using an LSTM to implement time-wise conditioning. |
| Software Dependencies | No | The paper states 'SCF layers are implemented via MADE (Germain et al., 2015).' but does not provide specific version numbers for any software libraries, programming languages, or tools used in the experiments. |
| Experiment Setup | No | The paper states 'Optimization and hyperparameter details are given in the Supplementary Materials.' indicating that these details are not provided in the main text. |