Multiple Hypothesis Dropout: Estimating the Parameters of Multi-Modal Output Distributions
Authors: David D. Nguyen, David Liebowitz, Salil S. Kanhere, Surya Nepal
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on supervised learning problems illustrate that our approach outperforms existing solutions for reconstructing multimodal output distributions. Additional studies on unsupervised learning problems show that estimating the parameters of latent posterior distributions within a discrete autoencoder significantly improves codebook efficiency, sample quality, precision and recall. |
| Researcher Affiliation | Collaboration | David D. Nguyen1,2,3, David Liebowitz1,4, Salil S. Kanhere1,3, Surya Nepal2,3 1UNSW Sydney 2CSIRO Data61 3Cybersecurity CRC 4Penten david.nguyen@data61.csiro.au, david.liebowitz@penten.com, salil.kanhere@unsw.edu.au, surya.nepal@data61.csiro.au |
| Pseudocode | No | No pseudocode or clearly labeled algorithm blocks were found. |
| Open Source Code | No | The paper does not provide a direct link to open-source code for the described methodology or an explicit statement of code release. |
| Open Datasets | Yes | Experiments utilize medium resolution image datasets: Fashion MNIST 28 28, Celeb A 64 64, and Image Net 64 64 (Xiao, Rasul, and Vollgraf 2017; Liu et al. 2018; Deng et al. 2009). |
| Dataset Splits | No | The paper mentions 'validation samples' when reporting FID scores, but does not provide specific details on dataset splits (e.g., percentages, sample counts) for training, validation, and testing. It uses standard benchmark datasets, but does not explicitly state how the splits were performed or if standard splits were used with citations. |
| Hardware Specification | No | The paper mentions 'GPU memory constraints' when discussing model comparisons but does not provide specific details on the hardware used for running experiments, such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies (e.g., libraries, frameworks, or programming languages) used in the experiments. |
| Experiment Setup | Yes | The mixture models employ three FFNs trained with a dropout rate of 0.5 and employ mixture coefficient layers. The mixture model of MH Dropout networks is trained using a subset ratio of 0.1. The number of hypotheses per pass was 64. Following existing practices (Esser, Rombach, and Ommer 2021), high down-sampling factors 𭟋= (14, 16, 32) and compression rates above 38.2 bits per dimension are applied. Token numbers vary by dataset, with primary tokens in the range 4 16 and a single secondary token per image. |