VAE Approximation Error: ELBO and Exponential Families
Authors: Alexander Shekhovtsov, Dmitrij Schlesinger, Boris Flach
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally investigate the detrimental effect in one case and show that a simpler but more consistent VAE can perform better in the other. 4 EXPERIMENTS |
| Researcher Affiliation | Academia | Alexander Shekhovtsov Czech Technical University in Prague shekhole@fel.cvut.cz Dmitrij Schlesinger Dresden University of Technology Dmytro.Shlezinger@tu-dresden.de Boris Flach Czech Technical University in Prague flachbor@fel.cvut.cz |
| Pseudocode | No | No pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | No | The paper mentions using "code of Inkawhich (2017)" and "code of Seitzer (2020)" for parts of their experiments, but does not provide any statement or link for the source code of their own methodology. |
| Open Datasets | Yes | The ground truth generative model is obtained by training a convolutional Generative Adversarial network (GAN) using code of Inkawhich (2017) on the Celeb A dataset Liu et al. (2015). |
| Dataset Splits | No | The dataset contains bag-of-words representations of documents and is split into a training set with 11269 documents and a test set with 7505 documents. The paper specifies training and test sets by count but does not mention a distinct validation set or cross-validation setup for hyperparameter tuning. |
| Hardware Specification | No | The authors gratefully acknowledge the Center for Information Services and HPC (ZIH) at TU Dresden for providing computing time. This mentions a computing center but does not provide specific hardware details (e.g., GPU/CPU models, memory). |
| Software Dependencies | Yes | using the code of Seitzer (2020). For this we generate 200k images from each model. The obtained values are given in Tab. 1. Fig. 4 shows images generated by the ground truth model and the two learned models. Maximilian Seitzer. pytorch-fid: FID Score for Py Torch. https://github.com/mseitzer/ pytorch-fid, August 2020. Version 0.1.1. |
| Experiment Setup | Yes | We train for 1000 epochs using 1-sample ARM and then for 500 more epochs using 10 samples for computing each gradient estimate with ARM. Adam optimizer with learning rate 0.001. For decoders with dhidden = 1, 2 the hidden layers contained 512 units. For the deep encoder e2 we used 2 hidden Re LU layers with 512 units each. |