Implicit Rank-Minimizing Autoencoder
Authors: Li Jing, Jure Zbontar, yann lecun
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically demonstrate IRMAE s regularization behavior through a synthetic dataset and show that it learns good representation with a much smaller latent dimension. Then we demonstrate superior representation learning performance of our method against a standard deterministic autoencoder and comparable performance to a variational autoencoder on MNIST dataset and Celeb A dataset through a variety of generative tasks, including interpolation, sample generation from noise, PCA interpolation in low dimension, and a downstream classification task. |
| Researcher Affiliation | Industry | Li Jing Facebook AI Research New York ljng@fb.com Jure Zbontar Facebook AI Research New York jzb@fb.com Yann Le Cun Facebook AI Research New York yann@fb.com |
| Pseudocode | No | The paper describes the method and architecture but does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statement about releasing source code or a link to a code repository. |
| Open Datasets | Yes | We train our model on the MNIST dataset [15] and the Celeb A dataset [16]. |
| Dataset Splits | No | Each plot in Figures 2 and 4 depicts singular values (sorted from large to small) of the covariance matrix of the latent variables z corresponding to examples in the validation set. IRMAE l = 2 yields excellent reconstructions on validation set. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU, CPU models, memory) used for running the experiments. |
| Software Dependencies | No | Each model is trained with an Adam optimizer with a learning rate of 0.001. |
| Experiment Setup | Yes | We set the latent dimension to 128/512 for the two datasets, respectively. We use 8/4 extra linear matrices for regularization in IRMAE, respectively. ... Each model is trained with an Adam optimizer with a learning rate of 0.001. ... This MLP head has two linear layers of hidden dimension 128, with Re LU activation. |