Do GANs learn the distribution? Some Theory and Empirics
Authors: Sanjeev Arora, Andrej Risteski, Yi Zhang
ICLR 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The current paper makes two contributions. (1) It proposes a novel test for estimating support size using the birthday paradox of discrete probability. Using this evidence is presented that well-known GANs approaches do learn distributions of fairly low support. (2) It theoretically studies encoder-decoder GANs architectures (e.g., Bi GAN/ALI), which were proposed to learn more meaningful features via GANs and (consequently) to also solve the mode-collapse issue. Our result shows that such encoder-decoder training objectives also cannot guarantee learning of the full distribution because they cannot prevent serious mode collapse. More seriously, they cannot prevent learning meaningless codes for data, contrary to usual intuition. |
| Researcher Affiliation | Academia | Sanjeev Arora Department of Computer Science Princeton University Princeton, NJ 08544, USA arora@cs.princeton.edu Andrej Risteski Applied Mathematics Department and IDSS Massachusetts Institute of Technology Cambridge, MA 02139, USA risteski@mit.edu Yi Zhang Department of Computer Science Princeton University Princeton, NJ 08544, USA y.zhang@cs.princeton.edu |
| Pseudocode | No | The paper does not contain explicitly labeled pseudocode or algorithm blocks. It has a figure depicting a network implementation but not a formal pseudocode section. |
| Open Source Code | No | The paper does not provide any explicit statement or link indicating that source code for the described methodology is publicly available. |
| Open Datasets | Yes | Our test were done using two datasets, Celeb A (faces) (Liu et al., 2015) and CIFAR-10 (Krizhevsky, 2009) . Note that Celeb A reasonably balanced, since the constructors intentionally made it unbiased (it contains ten thousand identities, each of which has twenty images). Also, we report in Appendix B results on the Bedroom dataset from the LSUN (Yu et al., 2015). |
| Dataset Splits | No | The paper does not explicitly state the training, validation, and test dataset splits with percentages or counts for reproducibility. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments (e.g., GPU models, CPU types). |
| Software Dependencies | No | The paper mentions models and architectures like DCGAN, MIX+ GAN, ALI, and Stacked GAN, but does not provide specific software dependencies or library versions (e.g., Python 3.x, PyTorch 1.x). |
| Experiment Setup | Yes | For fair comparison, we set the discriminator of ALI (or Bi GANs) to be roughly the same in size as that of the DCGAN model, since the results of Section 2.1.1 below suggests that the discriminator size has a strong effect on diversity of the learnt distribution. ... We build DCGANs with increasingly larger discriminators while fixing the other hyper-parameters. The discriminator used here is a 5-layer Convolutional Neural Network such that the number of output channels of each layer is 1 , 2 , 4 , 8 dim where dim is chosen to be 16, 24, . . . , 120, 128. Thus the discriminator size should be proportional to dim2. |