The Emergence of Reproducibility and Consistency in Diffusion Models
Authors: Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, Qing Qu
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We confirm this phenomenon through comprehensive experiments, implying that different diffusion models consistently reach the same data distribution and score function regardless of diffusion model frameworks, model architectures, or training procedures. |
| Researcher Affiliation | Academia | 1Department of Electrical and Computer Engineering, University of Michigan, Ann Arbor, Location, Ann Arbor, MI 48109-2122, USA. Correspondence to: Huijie Zhang <huijiezh@umich.edu>, Qing Qu <qingqu@umich.edu>. |
| Pseudocode | Yes | Algorithm 1 Determinsitic DPS with DPM-Solver. |
| Open Source Code | No | The paper discusses using 'author-released models' for existing diffusion models (e.g., in Appendix A, Table 1 notes 'all selected diffusion model architectures utilize the author-released models'), but it does not provide an explicit statement or link to open-source code developed by the authors for their methodology. |
| Open Datasets | Yes | We utilized denoising diffusion probabilistic models (DDPM) (Ho et al., 2020; Song et al., 2020a), consistency model (CT) (Song et al., 2023b), U-Vi T (Bao et al., 2023) trained on CIFAR-10 (Krizhevsky et al., 2009) dataset. |
| Dataset Splits | No | The paper mentions training on datasets and varying training data sizes, for instance, 'As for the dataset size, we select images from the CIFAR dataset, ranging from 26 to 215. Under each dataset size, different models are trained from the same subset of images.' However, it does not explicitly provide details about specific training/validation/test dataset splits or a methodology for them. |
| Hardware Specification | No | The paper states, 'Results presented in this paper were obtained using Cloud Bank, which is supported by the NSF under Award #1925001, and the authors acknowledge efficient cloud management framework Sky Pilot (Yang et al., 2023) for computing.' However, it does not provide specific hardware details such as exact GPU or CPU models, processor types, or memory amounts used for the experiments. |
| Software Dependencies | No | The paper mentions various software components and models like 'DDPMv4, DDPMv6 (Ho et al., 2020; Song et al., 2020a), Multistagev1 (Zhang et al., 2023), EDMv1 (Karras et al., 2022), UVi T (Bao et al., 2023), CT (Song et al., 2023b), Progressivev1 (Salimans & Ho, 2022))' and samplers like 'DPM-Solver (Lu et al., 2022), Heun-Solver (Karras et al., 2022), DDIM (Song et al., 2020a)'. However, it does not specify version numbers for any of these software packages or their underlying libraries (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | In terms of model size, we experiment with UNet-64, UNet-128, and UNet-256, where, for instance, UNet-64 indicates a UNet structure with an embedding dimension of 64. As for the dataset size, we select images from the CIFAR dataset, ranging from 26 to 215. |