Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
InsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Model
Authors: Sidi Lu, Tao Meng, Nanyun Peng
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on two lexically constrained text generation datasets and three machine translation datasets demonstrate INSNET s advantages over previous insertion-based methods in terms of training speed, inference efficiency, and generation quality. |
| Researcher Affiliation | Academia | Sidi Lu, Tao Meng, Nanyun Peng University of California, Los Angeles EMAIL |
| Pseudocode | No | The paper describes algorithms but does not contain a formally labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | No | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [No] They will be released upon camera ready. |
| Open Datasets | Yes | We follow the setup in Zhang et al. (2020) and address the unsupervised lexically constrained text generation problem on two datasets Yelp Review and News. ... Yelp Review dataset consists of 160K training sequences, 10K sequences for validation and 1k test sequences. |
| Dataset Splits | Yes | Yelp Review dataset consists of 160K training sequences, 10K sequences for validation and 1k test sequences. |
| Hardware Specification | Yes | All results are collected on a single NVIDIA RTX3090 GPU. |
| Software Dependencies | No | The paper mentions 'Huggingface for their great work of the Transformers project' and 'NVIDIA APEX library' but does not specify version numbers for any software dependencies. |
| Experiment Setup | Yes | For position prediction, we are inserting into slots with positions lying in the top 70% of the position distribution mass. For token prediction, we are doing top-{1, 1, 3, 3, 5} sampling over the vocabulary distribution. |