Gacs-Korner Common Information Variational Autoencoder
Authors: Michael Kleinman, Alessandro Achille, Stefano Soatto, Jonathan Kao
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we demonstrate that our formulation allows us to learn semantically meaningful common and unique factors of variation even on high-dimensional data such as images and videos. Moreover, on datasets where ground-truth latent factors are known, we show that we can accurately quantify the common information between the random variables. |
| Researcher Affiliation | Collaboration | 1University of California, Los Angeles 2AWS AI Labs 3Caltech |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks that are clearly labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | Yes | 2Code available at: https://github.com/mjkleinman/common-vae |
| Open Datasets | Yes | To empirically evaluate the ability to separate the common and unique latent factors we introduce two new datasets, which extend commonly used datasets for evaluating disentangled representations learning: d Sprites [5] and 3d Shapes [6]. |
| Dataset Splits | No | The paper states, 'We trained on 8000 samples and tested on 2000.' for evaluating information contained in the representation, but it does not specify train/validation/test splits, exact percentages, or sample counts for all experiments, or cite predefined splits. |
| Hardware Specification | Yes | Our experiments can be reproduced in approximately 3 days on a single GPU (g4dn instance). |
| Software Dependencies | No | The paper mentions using 'sklearn' and specific encoders/decoders but does not provide specific version numbers for these software components or any other libraries like Python or PyTorch. |
| Experiment Setup | Yes | We train our GK-VAE models with Adam using a learning rate of 0.001, unless otherwise stated. We set βu to be 10, βc to be 0.1 and λc = 0.1. We trained networks for 70 epochs, except for the MNIST experiments, where we trained for 50 (details in the Appendix). To improve optimization, we use the idea of free bits [33] and we set λfree-bits = 0.1. We used a batch size of 128 and we set λc = 0.1. |