Uncertainty Principles of Encoding GANs
Authors: Ruili Feng, Zhouchen Lin, Jiapeng Zhu, Deli Zhao, Jingren Zhou, Zheng-Jun Zha
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We prove three uncertainty principles of encoding GANs in practice: a) the perfect encoder and generator cannot be continuous at the same time, which implies that current framework of encoding GANs is illposed and needs rethinking; b) neural networks cannot approximate the underlying encoder and generator precisely at the same time, which explains why we cannot get perfect encoders and generators as promised in previous theories; c) neural networks cannot be stable and accurate at the same time, which demonstrates the difficulty of training and trade-off between fidelity and disentanglement encountered in previous works. Our work may eliminate gaps between previous theories and empirical results, promote the understanding of GANs, and guide network designs for follow-up works. This section presents a toy example to illustrate and support our theory. The toy example aims to learn the underlying encoder and generator between uniform distributions of supports of intrinsic dimensions in 1 or 2. |
| Researcher Affiliation | Collaboration | 1University of Science and Technology of China, Hefei, China. 2Key Lab. of Machine Perception (Mo E), School of EECS, Peking University, Beijing, China. 3Pazhou Lab, Guangzhou, China. 4Hong Kong University of Science and Technology, Hong Kong, China. 5Alibaba Group. |
| Pseudocode | No | The paper refers to 'Algorithm 1 of (Goodfellow et al., 2014)' but does not provide its own pseudocode or algorithm block. |
| Open Source Code | No | The paper does not include any statement or link indicating that its source code is open-source or publicly available. |
| Open Datasets | No | The paper uses a 'toy example' involving 'uniform distributions of supports of intrinsic dimensions in 1 or 2' and 'two-dimensional standard Gaussian N2(0, 1)'. These are described as self-generated distributions for a toy example, not named, publicly accessible datasets with concrete access information (link, DOI, or formal citation). |
| Dataset Splits | No | The paper describes training strategies for its toy example, such as 'concurrent training' and 'two phase training', but does not provide specific percentages or counts for training, validation, or test splits. It references training processes and objectives but not data partitioning. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU models, CPU types, or cloud computing instances used for running the experiments. |
| Software Dependencies | No | The paper mentions components like '3-layer MLPs with Leaky Relu activations', 'Jensen-Shannon divergence', 'Wasserstein distance', 'Bi GAN', and 'LIA', but does not list specific software, libraries, or programming language versions (e.g., Python 3.x, PyTorch 1.x) used for implementation. |
| Experiment Setup | Yes | The encoder, generator, and discriminator networks consist of 3-layer MLPs with Leaky Relu activations. The numbers of hidden units are 10, 100, and 10 for each MLP layer... We train the networks with both concurrent training and two phase training methods. For concurrent training, we use the objective (6) as in Bi GAN (Donahue et al., 2017); for two phase training, we use the objectives (8) and (10) with d = d Z + drecon, where d Z is the Jensen-Shannon divergence between the encoder output and the latent space, and drecon is the Jensen-Shannon divergence between the reconstructed data distribution and the real data distribution. For the zero-sum game in objectives (6), (8), (10), we solve it by the adversarial training process in Algorithm 1 of (Goodfellow et al., 2014). For each experiment setting, we further change the number of steps to apply to the discriminator... |