Understanding Hallucinations in Diffusion Models through Mode Interpolation
Authors: Sumukh K Aithal, Pratyush Maini, Zachary Lipton, J. Zico Kolter
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through experiments on 1D and 2D Gaussians, we show how a discontinuous loss landscape in the diffusion model s decoder leads to a region where any smooth approximation will cause such hallucinations. Through experiments on artificial datasets with various shapes, we show how hallucination leads to the generation of combinations of shapes that never existed. We extend the validity of mode interpolation in real-world datasets by explaining the unexpected generation of images with additional or missing fingers similar to those produced by popular textto-image generative models. Finally, we show that diffusion models in fact know when they go out of support and hallucinate. This is captured by the high variance in the trajectory of the generated sample towards the final few backward sampling steps. Using a simple metric to capture this variance, we can remove over 95% of hallucinations at generation time while retaining 96% of in-support samples in the synthetic datasets. We conclude our exploration by showing the implications of such hallucination (and its removal) on the collapse (and stabilization) of recursive training on synthetic data with experiments on MNIST and a 2D Gaussians dataset. |
| Researcher Affiliation | Collaboration | Sumukh K Aithal1 Pratyush Maini1,2 Zachary C. Lipton1 J. Zico Kolter1 Carnegie Mellon University1 Datology AI2 {saithal, pratyus2, zlipton, zkolter}@cs.cmu.edu |
| Pseudocode | No | No pseudocode or algorithm blocks are present in the paper. The methodology is described in prose. |
| Open Source Code | Yes | We release our code at https://github.com/locuslab/diffusion-model-hallucination. |
| Open Datasets | Yes | Through experiments on 1D and 2D Gaussians, we show how a discontinuous loss landscape in the diffusion model s decoder leads to a region where any smooth approximation will cause such hallucinations. We design a synthetic dataset called SIMPLE SHAPES, and train a diffusion model to learn its distribution. We train an ADM [29] on a dataset of high-quality hand images and show that the diffusion model generates hallucinated images of hands with additional fingers. We also run the recursive model training on the MNIST dataset [22]. |
| Dataset Splits | No | The paper does not explicitly state train/validation/test splits with percentages or counts for its datasets in the main text or Appendix A for its primary experiments. For example, for 1D Gaussian: 'We sample 50k training points from this true distribution'. For MNIST: 'At every generation, we generate 65k images and sample 60k images using the filtering mechanism.' No validation split is mentioned. |
| Hardware Specification | Yes | We run all our experiments of Nvidia RTX 2080 Ti and Nvidia A6000 GPUs. |
| Software Dependencies | No | Our implementations of the DDPM model is based on Py Torch [31]. |
| Experiment Setup | Yes | We run all our experiments for 10, 000 epochs with batch size of 10, 000. A linear noise schedule is used with starting noise β0 = 0.001 and the final noise β1 = 0.2. We use T = 1000 by default in our experiments (unless specified otherwise). ... Adam [21] with learning rate of 0.001 is used as the optimizer. For each generation, we train a class conditional DDPM with Classifier-Free Guidance [16] with T = 500 for 50 epochs. ... Adam [21] optimizer with learning rate of 1e-4 is used to train the network. |