Consistency Regularization for Generative Adversarial Networks
Authors: Han Zhang, Zizhao Zhang, Augustus Odena, Honglak Lee
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This section validates our proposed CR-GAN method. First we conduct a large scale study to compare consistency regularization to existing GAN regularization techniques (Kodali et al., 2017; Gulrajani et al., 2017; Roth et al., 2017) for several GAN architectures, loss functions and other hyperparameter settings. We then apply consistency regularization to a state-of-the-art GAN model (Brock et al., 2018) and demonstrate performance improvement. Finally, we conduct ablation studies to investigate the importance of various design choices and hyper-parameters. |
| Researcher Affiliation | Industry | Han Zhang, Zizhao Zhang, Augustus Odena, Honglak Lee Google Research {zhanghan,zizhaoz,augustusodena,honglak}@google.com |
| Pseudocode | Yes | Algorithm 1 Consistency Regularized GAN (CR-GAN). We use λ = 10 by default. |
| Open Source Code | Yes | All our experiments are based on the open-source code from Compare GAN (Kurach et al., 2019), which is available at https://github.com/google/compare_gan. |
| Open Datasets | Yes | We validate our proposed method on three datasets: CIFAR-10 (Krizhevsky, 2009), CELEBA-HQ-128 (Karras et al., 2018), and Image Net-2012 (Russakovsky et al., 2015). |
| Dataset Splits | No | The paper specifies training and testing set sizes for CIFAR-10 (50K training, 10K testing) and Celeb A (27K training, 3K testing), and mentions Image Net-2012, but does not explicitly define a separate validation dataset split or state a methodology for hyperparameter tuning that uses a validation set. |
| Hardware Specification | Yes | Here we show the actual training speed of discriminator updates for SNDCGAN on CIFAR-10 with NVIDIA Tesla V100. |
| Software Dependencies | No | The paper mentions using the 'Adam optimizer' and that experiments are 'based on the open-source code from Compare GAN', but does not provide specific version numbers for software libraries, frameworks (e.g., PyTorch, TensorFlow), or programming languages used. |
| Experiment Setup | Yes | For optimization, we use the Adam optimizer with batch size of 64 for all our experiments. We stop training after 200k generator update steps for CIFAR-10 and 100k steps for Celeb A. Appendix A (Table A1) provides hyper-parameters of the optimizer including learning rate (lr), Adam β1 and β2, and number of discriminator iterations per generator update (NDis) across 7 different settings. |