Neural Discrete Representation Learning
Authors: Aaron van den Oord, Oriol Vinyals, koray kavukcuoglu
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | As a first experiment we compare VQ-VAE with normal VAEs (with continuous variables), as well as VIMCO [28] with independent Gaussian or categorical priors. We train these models using the same standard VAE architecture on CIFAR10, while varying the latent capacity (number of continuous or discrete latent variables, as well as the dimensionality of the discrete space K). |
| Researcher Affiliation | Industry | Aaron van den Oord Deep Mind avdnoord@google.com Oriol Vinyals Deep Mind vinyals@google.com Koray Kavukcuoglu Deep Mind korayk@google.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks clearly labeled as such. |
| Open Source Code | No | The paper provides a URL for audio samples: 'https://avdnoord.github.io/homepage/vqvae/', but does not provide concrete access to the source code for the methodology described in the paper. |
| Open Datasets | Yes | In our work, we use three complex image datasets (CIFAR10, Image Net, and Deep Mind Lab) and a raw speech dataset (VCTK). |
| Dataset Splits | No | The paper mentions training models on datasets like CIFAR10 and evaluating performance, but it does not specify exact train/validation/test dataset splits by percentages or sample counts, nor does it explicitly refer to predefined splits with citations for reproducibility. |
| Hardware Specification | No | The paper describes the model architecture and training process, but does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions using the ADAM optimizer, but it does not provide specific version numbers for ancillary software details (e.g., library or solver names with version numbers like Python 3.8, CPLEX 12.4) needed to replicate the experiment. |
| Experiment Setup | Yes | We use the ADAM optimiser [21] with learning rate 2e-4 and evaluate the performance after 250,000 steps with batch-size 128. For VIMCO we use 50 samples in the multi-sample training objective. |