InsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Model
Authors: Sidi Lu, Tao Meng, Nanyun Peng
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on two lexically constrained text generation datasets and three machine translation datasets demonstrate INSNET s advantages over previous insertion-based methods in terms of training speed, inference efficiency, and generation quality. |
| Researcher Affiliation | Academia | Sidi Lu, Tao Meng, Nanyun Peng University of California, Los Angeles {sidilu, tmeng, violetpeng}@cs.ucla.edu |
| Pseudocode | No | The paper describes algorithms but does not contain a formally labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | No | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [No] They will be released upon camera ready. |
| Open Datasets | Yes | We follow the setup in Zhang et al. (2020) and address the unsupervised lexically constrained text generation problem on two datasets Yelp Review and News. ... Yelp Review dataset consists of 160K training sequences, 10K sequences for validation and 1k test sequences. |
| Dataset Splits | Yes | Yelp Review dataset consists of 160K training sequences, 10K sequences for validation and 1k test sequences. |
| Hardware Specification | Yes | All results are collected on a single NVIDIA RTX3090 GPU. |
| Software Dependencies | No | The paper mentions 'Huggingface for their great work of the Transformers project' and 'NVIDIA APEX library' but does not specify version numbers for any software dependencies. |
| Experiment Setup | Yes | For position prediction, we are inserting into slots with positions lying in the top 70% of the position distribution mass. For token prediction, we are doing top-{1, 1, 3, 3, 5} sampling over the vocabulary distribution. |