Language GANs Falling Short
Authors: Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, Laurent Charlin
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using our evaluation approach (Section 3), we examine several recent GAN text generation models and compare against an MLE baseline. The experiments consist of two parts: synthetic data generation and long-text generation. We provide strong empirical evidence for both types of data that MLE trained models reliably outperform textual GANs in the quality-diversity space. |
| Researcher Affiliation | Collaboration | Massimo Caccia Mila, Universit e de Montr eal massimo.p.caccia@gmail.com Lucas Caccia Mila, Mc Gill University lucas.page-caccia@mail.mcgill.ca William Fedus Mila, Universit e de Montr eal Google Brain, Montr eal Hugo Larochelle Google Brain, Montr eal Mila, Universit e de Montr eal Canada CIFAR AI Chair Jo elle Pineau Mila, Mc Gill University Facebook AI Research, Montr eal Canada CIFAR AI Chair Laurent Charlin Mila, HEC Montr eal Canada CIFAR AI Chair |
| Pseudocode | No | The paper describes algorithms and methods in prose but does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code to reproduce experiments is available at github.com/pclucas14/GansFallingShort |
| Open Datasets | Yes | The EMNLP2017 News dataset", "IMDB movie reviews (Maas et al., 2011) and Wiki Text-103 (Merity et al., 2016) datasets", "Image COCO dataset (Lin et al., 2014) |
| Dataset Splits | No | The paper mentions using "held-out validation perplexities" and "held-out data" for evaluation, but does not provide specific details or ratios for training, validation, and test dataset splits. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as CPU/GPU models, memory, or cloud instance specifications used for running experiments. |
| Software Dependencies | No | The paper describes the models implemented (e.g., RL-GAN, MLE model, LSTM oracle) but does not specify any software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We ran a hyperparameter search of 300 trials encompassing all possible combinations of said functionalities.", "fixed LSTM oracle Yu et al. (2017) with a hidden dimension of 32", "softmax(ot W/α) |