Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Disentanglement via Latent Quantization
Authors: Kyle Hsu, William Dorrell, James Whittington, Jiajun Wu, Chelsea Finn
NeurIPS 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the broad applicability of this approach by adding it to both basic data-reconstructing (vanilla autoencoder) and latent-reconstructing (Info GAN) generative models. For reliable evaluation, we also propose Info MEC, a new set of metrics for disentanglement that is cohesively grounded in information theory and fixes well-established shortcomings in previous metrics. Together with regularization, latent quantization dramatically improves the modularity and explicitness of learned representations on a representative suite of benchmark datasets. In particular, our quantized-latent autoencoder (QLAE) consistently outperforms strong methods from prior work in these key disentanglement properties without compromising data reconstruction. |
| Researcher Affiliation | Academia | Kyle Hsu Will Dorrell James C. R. Whittington Jiajun Wu Chelsea Finn Stanford University University College London Oxford University |
| Pseudocode | Yes | Algorithm 1 contains pseudocode for latent quantization and computing the quantization and commitment losses. Appendix A presents pseudocode for training a quantized-latent autoencoder (QLAE) in Algorithm 2 and a quantized-latent Info WGAN-GP [12, 1, 23] in Algorithm 3. |
| Open Source Code | Yes | Code for models and Info MEC metrics: https://github.com/kylehkhsu/latent_quantization. |
| Open Datasets | Yes | We benchmark on four established datasets: Shapes3D [9], MPI3D [20], Falcor3D [53], and Isaac3D [53]. |
| Dataset Splits | No | The paper states 'We train on the entire dataset then evaluate on 10,000 i.i.d. samples' which describes a test set, and mentions hyperparameter tuning, but does not explicitly provide details on a validation dataset split. |
| Hardware Specification | No | The paper does not provide specific details on the hardware used, such as GPU or CPU models. |
| Software Dependencies | No | The paper acknowledges the use of several open-source software packages (e.g., NumPy, JAX, scikit-learn) but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | We fix the number of latents in all methods to twice the number of sources. For quantized-latent models, we fix nv = 10 discrete values per codebook. We tune one key regularization hyperparameter per method per dataset with a thorough sweep (Table 11, Appendix C.2). Appendices C.2 and C.3 provide specific values for learning rates, batch sizes, update steps, loss weights, and network architectures. |