Plug and Play Language Models: A Simple Approach to Controlled Text Generation
Authors: Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, Rosanne Liu
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Model samples demonstrate control over a range of topics and sentiment styles, and extensive automated and human annotated evaluations show attribute alignment and fluency. |
| Researcher Affiliation | Collaboration | Sumanth Dathathri CMS, Caltech Andrea Madotto HKUST Janice Lan Uber AI Jane Hung Uber AI Eric Frank Uber AI Piero Molino Uber AI Jason Yosinski Uber AI Rosanne Liu |
| Pseudocode | No | The paper describes the method using mathematical equations and descriptive text, but does not include any clearly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | Code for the experiments is available at: https://github.com/uber-research/PPLM. |
| Open Datasets | Yes | The sentiment discriminator here distinguishes sentiment between POSITIVE and NEGATIVE and is trained on the SST-5 dataset (Socher et al., 2013). |
| Dataset Splits | Yes | The sentiment discriminator here distinguishes sentiment between POSITIVE and NEGATIVE and is trained on the SST-5 dataset (Socher et al., 2013). |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware used for running experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper mentions using a GPT-2 model and a PyTorch transformer, but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | Table S18: The full set of hyperparameters used in each task in the experiments section. Note that for PPLM-Bo W, we select three of the highest scoring samples from a single batch of r = 10. For PPLM-Discrim, we get 1 sample per batch, across 3 batches of r = 10. Method Type Attribute Hyperparameters PPLM-Bo W Politics, Legal, Computers, Space, Science, Military m = 3, λkl = 0.01, α = 0.01, γ = 1.5, γgm = 0.9, r = 10, τ = 0.85 PPLM-Bo W Religion m = 3, λkl = 0.01, α = 0.01, γ = 1.5, γgm = 0.8, r = 10, τ = 0.85 PPLM-Discrim POSITIVE, NEGATIVE m = 10, λkl = 0.01, α = 0.03, γ = 1.0, γgm = 0.95, r = 10, τ = 0.9 PPLM-Discrim Detoxicification m = 10, λkl = 0.01, α = 0.02, γ = 1.0, γgm = 0.9, r = 1, τ = 0 |