Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias
Authors: Frederic Koehler, Viraj Mehta, Chenghui Zhou, Andrej Risteski
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we revisit this setting and explore the behaviour of both the VAE loss, and the training dynamics. Through a combination of theory and experiments we show that: |
| Researcher Affiliation | Academia | Frederic Koehler 1, Viraj Mehta 2, Chenghui Zhou 3, and Andrej Risteski3 1Department of Computer Science, Stanford University, fkoehler@stanford.edu 2Robotics Institute, Carnegie Mellon University, virajm@cs.cmu.edu 3Machine Learning Department, Carnegie Mellon University, {chenghuz, aristesk}@andrew.cmu.edu |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code (specific repository link, explicit code release statement, or code in supplementary materials) for the methodology described in this paper. |
| Open Datasets | No | The paper describes how synthetic data is generated (e.g., "To generate a sample point x for the data distribution, we will sample z N(0, Ir ), and output x = f(z)" and "We choose a ground truth linear transformation matrix A... the data is generated as Az, z N(0, Ir )") but does not provide a link, DOI, repository, or formal citation for a publicly available or open dataset. |
| Dataset Splits | No | The paper describes experiments with synthetic data but does not specify exact split percentages, absolute sample counts, or reference predefined splits with citations for training, validation, and test sets to reproduce the data partitioning. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers like Python 3.8, CPLEX 12.4) needed to replicate the experiment. |
| Experiment Setup | Yes | Parameterization of the trained model: For the model we are training, the generator will sample z N(0, Ir) and output x N(f(z), ϵ2I), for trainable f, ϵ; the encoder given input x will output z N(g(x), D), where D Rr r is a diagonal matrix, and g, D are trainable. In the linear case, f, g will be parameterized as matrices A, B; in the nonlinear case, several different parameterizations will be considered. ... We used 3 layers of 200 hidden units to parameterize our encoder and decoder networks. ... All VAEs are trained with a 20-dimensional latent space. |