Overlooked Implications of the Reconstruction Loss for VAE Disentanglement
Authors: Nathan Michlo, Richard Klein, Steven James
IJCAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We verify this by constructing an example dataset that prevents disentanglement in state-of-the-art frameworks while maintaining human-intuitive ground-truth factors. Finally, we re-enable disentanglement by designing an example reconstruction loss that is once again able to perceive the ground-truth factors. Our findings demonstrate the subjective nature of disentanglement... We compute distance matrices over a trained β-VAE at various levels of the network, including the representation layer and reconstructions. At each level of the VAE, the learnt distances all correspond to the original perceived distances already present within the dataset, see Figure 4. |
| Researcher Affiliation | Academia | Nathan Michlo, Richard Klein, Steven James University of the Witwatersrand, Johannesburg, South Africa nathan.michlo1@students.wits.ac.za, {richard.klein, steven.james}@wits.ac.za |
| Pseudocode | No | The information is insufficient. The paper does not include any figure, block, or section labeled 'Pseudocode' or 'Algorithm', nor does it present structured steps for a method formatted like code. |
| Open Source Code | Yes | We contribute Disent, a general PyTorch [Paszke et al., 2017] disentanglement framework, with common models, metrics, and datasets.2 Disent framework repository: https://github.com/nmichlo/disent. Code is provided under the MIT license. |
| Open Datasets | Yes | Consider the 3D Shapes dataset [Burgess and Kim, 2018]... d Sprites: Disentanglement testing sprites dataset. https://github.com/deepmind/ dsprites-dataset, 2017. Accessed: 2023-01-01. Taking inspiration from the chess piece example, we design a synthetic adversarial dataset called XYSquares (See Figure 5) |
| Dataset Splits | No | The information is insufficient. The paper states 'We perform an extensive hyper-parameter grid search for existing frameworks and datasets before running our own experiments.' but does not specify exact split percentages, sample counts, or reference predefined splits for reproducibility. |
| Hardware Specification | No | Computations were performed using the High Performance Computing infrastructure provided by the Mathematical Sciences Support unit at the University of the Witwatersrand. |
| Software Dependencies | No | The information is insufficient. The paper mentions 'PyTorch [Paszke et al., 2017]' but does not provide a specific version number for PyTorch or any other software components used for the experiments. |
| Experiment Setup | Yes | We use the same Adam [Kingma and Ba, 2015] optimiser and convolutional neural architecture as Burgess et al.[2017]. ... Finally, we perform an extensive hyper-parameter grid search for existing frameworks and datasets before running our own experiments. Hyperparameters include the learning rate, size of the latent dimension, training steps, batch size and β values. |