Bit Prioritization in Variational Autoencoders via Progressive Coding
Authors: Rui Shu, Stefano Ermon
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We first show empirically that conventionally-trained HVAEs are not good progressive coders. We then propose a simple method that constrains the hierarchical representations to prioritize the encoding of information beneficial for lossy compression, and show that this modification leads to improved sample quality. Our work lends further support to the progressive coding hypothesis and demonstrates that this hypothesis should be exploited when designing variational autoencoders. |
| Researcher Affiliation | Academia | 1Stanford Computer Science. |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link regarding the availability of its source code. |
| Open Datasets | Yes | We conduct the experiment on SVHN, Celeb A, and LSUNBedrooms by first training a VDVAE (Child, 2020). |
| Dataset Splits | No | The paper mentions using SVHN, Celeb A, and LSUN-Bedrooms datasets but does not explicitly provide information on training, validation, and test splits (e.g., percentages or sample counts). |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments. |
| Software Dependencies | No | The paper mentions using the “default official VDVAE implementation” and specifies hyperparameters, but does not list specific software dependencies with version numbers (e.g., Python 3.x, PyTorch 1.x). |
| Experiment Setup | Yes | Across all experiments, we use the default official VDVAE implementation with the following specifications applied to the hyperparameter file hps.py svhnx.dec_blocks = "1x1,4m1,4x2,8m4,8x5,16m8,16x10,32m16,32x21" svhnx.enc_blocks = "32x11,32d2,16x6,16d2,8x6,8d2,4x3,4d4,1x3" svhnx.n_batch = 12 celeba.dec_blocks = "1x1,4m1,4x2,8m4,8x5,16m8,16x10,32m16,32x21,64m32,64x5" celeba.enc_blocks = "64x5,64d2,32x11,32d2,16x6,16d2,8x6,8d2,4x3,4d4,1x3" celeba.n_batch = 6 lsun.dec_blocks = "1x1,4m1,4x2,8m4,8x5,16m8,16x10,32m16,32x21,64m32,64x5" lsun.enc_blocks = "64x5,64d2,32x11,32d2,16x6,16d2,8x6,8d2,4x3,4d4,1x3" lsun.n_batch = 6 # The following is applied to all datasets: parser.add_argument( --width , type=int, default=384) parser.add_argument( --zdim , type=int, default=16) parser.add_argument( --lr , type=float, default=0.0002) parser.add_argument( --skip_threshold , type=float, default=500.0) parser.add_argument( --skip_threshold , type=float, default=500.0) parser.add_argument( --ema_rate , type=float, default=0.9999) parser.add_argument( --adam_beta1 , type=float, default=0.9) parser.add_argument( --adam_beta2 , type=float, default=0.9) parser.add_argument( --warmup_iters , type=float, default=0) |