Disentangling Learning Representations with Density Estimation
Authors: Eric Yeats, Frank Y Liu, Hai Li
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our experiments, GCAE achieves highly competitive and reliable disentanglement scores compared with state-of-the-art baselines. and Experiments1 which demonstrate competitive performance of GCAE against leading disentanglement baselines on multiple datasets using existing metrics |
| Researcher Affiliation | Collaboration | Eric Yeats1 Frank Liu2 Hai Li1 1Department of Electrical and Computer Engineering, Duke University 2Computer Science and Mathematics Division, Oak Ridge National Laboratory {eric.yeats, hai.li}@duke.edu liufy@ornl.gov |
| Pseudocode | No | The paper describes algorithms and methods using mathematical notation and text, but it does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | 1Code available at https://github.com/ericyeats/gcae-disentanglement |
| Open Datasets | Yes | We consider two datasets which cover different data modalities. The Beamsynthesis dataset Yeats et al. (2022) is a collection of 360 timeseries data from a linear particle accelerator beamforming simulation. The waveforms are 1000 values long and are made of two independent data generating factors: duty cycle (continuous) and frequency (categorical). The d Sprites dataset Matthey et al. (2017) is a collection of 737280 synthetic images of simple white shapes on a black background. |
| Dataset Splits | Yes | For Beamsynthesis, we calculate MIG on the full dataset using a histogram estimate of the latent space with 50 bins (evenly spaced maximum to minimum). For d Sprites, we calculate MIG using 10000 samples, and we use 20 histogram bins following Locatello et al. (2019). and For Beamsynthesis, we train the majority-vote classifier on 1000 training points and evaluate on 200 separate points. For d Sprites, we train the majority-vote classifier on 5000 training points and evaluate on 1000 separate points. and For Beamsynthesis, we use a training size of 240 and a test size of 120. For d Sprites, we use a training size of 5000 and a test size of 1000. and For Beamsynthesis, we use 240 training points and 120 testing points. For d Sprites, we use 5000 training points and 1000 testing points. |
| Hardware Specification | Yes | All experiments are run using the Py Torch framework Paszke et al. (2019) using 4 NVIDIA Tesla V100 GPUs |
| Software Dependencies | No | The paper mentions using the 'Py Torch framework Paszke et al. (2019)' but does not specify a version number for PyTorch or any other software libraries required for replication. |
| Experiment Setup | Yes | In all experiments, the GCAE AE and discriminator learning rates are 5e 5 and 2e 4, respectively. The VAE learning rate is 1e 4 and the Factor VAE discriminator learning rate is 2e 4. All methods use the Adam optimizer with (β1, β2) = (0.9, 0.999) for the AE subset of parameters and (β1, β2) = (0.5, 0.9) for the discriminator(s) subset of parameters (if applicable). The number of discriminator updates per AE update k is set to 5 when m = 10 and 10 when m = 20. All discriminators are warmed up with 500 batches before training begins to ensure they approximate a valid density. (From Appendix B) and Table 1: MLP Architecture... Batch Size=64... 2000 Iterations... Batch Size=256... 20000 Iterations |