ContraGAN: Contrastive Learning for Conditional Image Generation
Authors: Minguk Kang, Jaesik Park
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform image generation experiments on CIFAR10 [24], Tiny Image Net [25], and Image Net [18] datasets using various backbone architectures, such as DCGAN [2], Res GAN [26, 16], and Big GAN [6] equipped with spectral normalization [4]. Through exhaustive experiments, we verify that the proposed Contra GAN improves the state-of-the-art-models by 7.3% and 7.7% on Tiny Image Net and Image Net datasets respectively, in terms of Frechet Inception Distance (FID) [27]. Also, Contra GAN gives comparable results (1.3% lower FID) on CIFAR10 with the art model [6]. Since Contra GAN can learn plentiful data-to-data relations from a properly sized batch, it reduces FID significantly without hard negative and positive mining. Furthermore, we experimentally show that 2C loss alleviates the overfitting problem of the discriminator. In the ablation study, we demonstrate that Contra GAN can benefit from consistency regularization [7] that uses data augmentations. |
| Researcher Affiliation | Academia | Minguk Kang Jaesik Park Graduate School of Artificial Intelligence POSTECH {mgkang, jaesik.park}@postech.ac.kr |
| Pseudocode | Yes | Algorithm 1 : Training Contra GAN |
| Open Source Code | Yes | For a fair comparison, we re-implement twelve state-of-the-art GANs using the Py Torch library. The software package is available at https://github.com/POSTECH-CVLab/Py Torch-Studio GAN. |
| Open Datasets | Yes | We perform conditional image generation experiments with CIFAR10 [24], Tiny Image Net [25], and Image Net [18] datasets to compare the proposed approach with other approaches. CIFAR10 [24] is a widely used benchmark dataset... Tiny Image Net [25] provides 120,000 color images in total... Image Net [18] provides 1,281,167 and 50,000 color images for training and validation respectively... |
| Dataset Splits | Yes | Tiny Image Net [25]...Each category has 600 images divided into 500, 50, and 50 samples for training, validation, and testing, respectively. Image Net [18] provides 1,281,167 and 50,000 color images for training and validation respectively, and the dataset consists of 1,000 categories. For the experiments using Tiny Image Net and Image Net, we use the validation set with the same amount of generated images. |
| Hardware Specification | No | The paper mentions "The supercomputing resources for this work was partially supported by Grand Challenging Project of Supercomuting Bigdata Center, DGIST." This is a general statement about computing resources but lacks specific hardware details such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | The paper states "we re-implement twelve state-of-the-art GANs using the Py Torch library." and "All FID values reported in our paper are calculated using the Py Torch FID implementation [46]." However, it does not specify the version numbers for PyTorch or any other relevant software libraries or dependencies, which is necessary for reproducibility. |
| Experiment Setup | Yes | Algorithm 1 : Training Contra GAN Input: Learning rate: α1, α2. Adam hyperparameters [41]: β1, β2. Batch size: m. Temperature: t. We do a grid search to find a proper temperature (t) used in Eq. 8 and experimentally found that the temperature of 1.0 gives the best results. For the dimension of the projection layer, we select 512 for CIFAR10, 768 for Tiny Image Net, and 1,024 for Image Net experiments. We conduct Image Net [18] experiments with a batch size of 256. The linear projection with batch size 64 for CIFAR10 and 1,024 for Tiny Image Net performs the best. |