DVAE++: Discrete Variational Autoencoders with Overlapping Transformations
Authors: Arash Vahdat, William Macready, Zhengbing Bian, Amir Khoshaman, Evgeny Andriyash
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on several benchmarks show that overlapping transformations outperform other recent continuous relaxations of discrete latent variables including Gumbel-Softmax (Maddison et al., 2016; Jang et al., 2016), and discrete variational autoencoders (Rolfe, 2016). |
| Researcher Affiliation | Industry | 1Quadrant.ai, D-Wave Systems Inc., Burnaby, BC, Canada. |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any statements about providing open-source code or links to a code repository. |
| Open Datasets | Yes | We test dynamically binarized MNIST (Le Cun et al., 1998) and Caltech-101 silhouettes (Marlin et al., 2010). All datasets have 28 28 binary pixel images. We also evaluate DVAE++ on the CIFAR10 dataset, which consists of 32 32 pixel natural images. |
| Dataset Splits | Yes | The inverse temperature β for smoothing is annealed linearly during training with initial and final values chosen using cross validation from {5, 6, 7, 8} and {12, 14, 16, 18} respectively. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments. |
| Software Dependencies | No | The paper mentions the use of the Adam optimizer but does not specify version numbers for any software dependencies like programming languages or libraries. |
| Experiment Setup | Yes | We used Adam with its default parameters except for which is set to 10 3. The learning rate is selected from the set {1 10 4, 5 10 4}. The inverse temperature β for smoothing is annealed linearly during training with initial and final values chosen using cross validation from {5, 6, 7, 8} and {12, 14, 16, 18} respectively. For all the models, we used 16 layers of local latent variables each with 32 random variables at each spatial location. For the RBM global variables, we used 16 binary variables for all the binary datasets and 128 binary variables for CIFAR10. We cross-validated the number of the hierarchical layers in the inference model for the global variables from the set {1, 2, 4}. |