Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
Authors: Sharon Zhou, Eric Zelikman, Fred Lu, Andrew Y. Ng, Gunnar E. Carlsson, Stefano Ermon
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To illustrate the effectiveness and applicability of our method, we empirically evaluate several state-of-the-art models across multiple datasets. We find that our method ranks models similarly to existing methods. |
| Researcher Affiliation | Academia | Sharon Zhou, Eric Zelikman, Fred Lu, Andrew Y. Ng, Gunnar Carlsson, Stefano Ermon Computer Science & Math Departments, Stanford University {sharonz, ezelikman, fredlu, ang, ermon}@cs.stanford.edu, carlsson@stanford.edu |
| Pseudocode | Yes | Algorithm 1: Procedure for producing W. RLTs on generated images |
| Open Source Code | Yes | We make our code publicly available at https://github.com/stanfordmlgroup/disentanglement. |
| Open Datasets | Yes | Datasets. We present empirical results on three datasets: (1) d Sprites (Matthey et al., 2017) is a canonical disentanglement dataset whose five generating factors {shape, scale, orientation, x-position, y-position} are complete and independent, i.e. they fully describe all combinations in the dataset; (2) Celeb A is a popular dataset for disentanglement and image generation, and is comprised of over 202K human faces, which we align and crop to be 64 64 px (Liu et al., 2015). There are also 40 attribute labels for each image; and (3) Celeba-HQ, a higher resolution subset of Celeb A consisting of 30K images (Karras et al., 2018). |
| Dataset Splits | No | The paper mentions using datasets for evaluation but does not specify any training, validation, or test splits, or cross-validation setup. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments. |
| Software Dependencies | No | The paper does not list specific software components with their version numbers. While VGG16 is mentioned, it's a pre-trained model used for embedding, not a software dependency with a version. |
| Experiment Setup | Yes | For the embedding function in Algorithms 1 and 2, we used a pretrained VGG16 (Simonyan and Zisserman, 2015) with the last 3 layers removed to embed image samples into 64 feature dimensions. Additional training details are in Appendix G. |