Improving GAN Training with Probability Ratio Clipping and Sample Reweighting
Authors: Yue Wu, Pan Zhou, Andrew G. Wilson, Eric Xing, Zhiting Hu
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on three unsupervised generation tasks, including image generation, text generation, and text style transfer. The three tasks apply GANs to model different data modalities, namely, image, text, and neural hidden representations, respectively. Our approach consistently offers improvement over the state-of-the-arts on all tasks. See appendix for all experimental details. |
| Researcher Affiliation | Collaboration | Yue Wu1, Pan Zhou2, Andrew Gordon Wilson3, Eric P. Xing1,4, Zhiting Hu1,5 1Carnegie Mellon University, 2National University of Singapore, 3New York University 4Petuum Inc., 5UC San Diego |
| Pseudocode | Yes | Algorithm 1 GAN Training with Probability Ratio Clipping and Sampling Re-weighting |
| Open Source Code | Yes | Code available at: github.com/Holmeswww/PPOGAN |
| Open Datasets | Yes | We first use the popular CIFAR-10 benchmark for evaluation... CIFAR-10 [28] contains 50K images of sizes 32 × 32. ... We then evaluate our method on the EMNLP2017 WMT News, a large real text data used for text GAN studies [16, 35]. ... We use the standard Yelp review dataset, and the ground truth output text provided by [30]. |
| Dataset Splits | No | The dataset consists of 270K/10K training/test sentences with a maximum length of 51 and a vocabulary size of 5,255. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware specifications (e.g., GPU models, CPU types, memory) used to conduct the experiments. |
| Software Dependencies | No | The paper mentions using specific algorithms and architectures such as 'Gumbel-softmax approximation [25, 31]' and implementing based on 'Rel GAN [35] architecture', but it does not specify software versions for any frameworks (e.g., PyTorch, TensorFlow) or libraries used. |
| Experiment Setup | Yes | For each iteration, we update both generator and discriminator for 5 times. ... Our full approach and clipping only use an update ratio of 5:5, because the probability ratio clipping that discourages large generator updates allows us to update the generator more frequently, which is desirable. ... the generator is initialized with maximum likelihood (MLE) pre-training. |