Large Scale Adversarial Representation Learning
Authors: Jeff Donahue, Karen Simonyan
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We extensively evaluate the representation learning and generation capabilities of these Big Bi GAN models, demonstrating that these generation-based models achieve the state of the art in unsupervised representation learning on Image Net, as well as in unconditional image generation. [...] We perform a thorough empirical analysis and ablation study of model design choices. |
| Researcher Affiliation | Industry | Jeff Donahue Deep Mind jeffdonahue@google.com Karen Simonyan Deep Mind simonyan@google.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | We open source pretrained Big Bi GAN models on Tensor Flow Hub. Models available at https://tfhub.dev/s?publisher=deepmind&q=bigbigan, with a Colab notebook demo at https://colab.research.google.com/github/tensorflow/hub/blob/master/ examples/colab/bigbigan_with_tf_hub.ipynb. |
| Open Datasets | Yes | We train a Big Bi GAN on unlabeled Image Net, freeze its learned representation, and then train a linear classifier on its outputs, fully supervised using all of the training set labels. Most of our experiments follow the standard protocol used to evaluate unsupervised learning techniques, first proposed in [34]. |
| Dataset Splits | Yes | Big Bi GAN results are selected with early stopping based on highest accuracy on our trainval subset of 10K training set images. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Tensor Flow Hub' as the platform where models are available and a 'Colab notebook demo', but does not provide specific software dependencies like library names with version numbers (e.g., 'TensorFlow 2.x', 'PyTorch 1.x'). |
| Experiment Setup | Yes | We start with a relatively fully-fledged version of the model at 128 128 resolution (row Base), with the G architecture and the F component of D taken from the corresponding 128 128 architectures in Big GAN, including the skip connections and shared noise embedding proposed in [1]. z is 120 dimensions, split into six groups of 20 dimensions fed into each of the six layers of G as in [1]. The remaining components of D H and J are 8-layer MLPs with Res Net-style skip connections (four residual blocks with two layers each) and size 2048 hidden layers. The E architecture is the Res Net-v2-50 Conv Net originally proposed for image classification in [13], followed by a 4-layer MLP (size 4096) with skip connections (two residual blocks) after Res Net s globally average pooled output. [...] simply using a 10 higher learning rate for E dramatically accelerates training and improves final representation learning results. |