Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structure
Authors: Xiang Li, Yixiang Dai, Qing Qu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically demonstrate that this inductive bias is a unique property of diffusion models in the generalization regime... Through various experiments and theoretical investigation, we show that Diffusion models in the generalization regime have inductive bias towards learning the Gaussian structures of the dataset. |
| Researcher Affiliation | Academia | Xiang Li1, Yixiang Dai1, Qing Qu1 1Department of EECS, University of Michigan, forkobe@umich.edu, yixiang@umich.edu, qingqu@umich.edu |
| Pseudocode | Yes | Algorithm 1 Linear Distillation |
| Open Source Code | Yes | The code and instructions for reproducing the experiment results will be made available in the following link: https://github.com/Morefre/Understanding-Generalizability-of Diffusion-Models-Requires-Rethinking-the-Hidden-Gaussian-Structure. |
| Open Datasets | Yes | 1For example, FFHQ [26], CIFAR-10 [27], AFHQ [28] and LSUN-Churches [29]. |
| Dataset Splits | No | The paper discusses training on various dataset sizes and measures like FID convergence but does not provide explicit details about train/validation/test dataset splits (percentages or counts). |
| Hardware Specification | Yes | All the diffusion models in the experiments are trained on A100 GPUs provided by NCSA Delta GPU [33]. |
| Software Dependencies | No | Additionally, we employ the Adam optimizer [35] for updates. |
| Experiment Setup | Yes | Following the EDM training configuration [4], we set the noise levels σ(t) within the continuous range [0.002,80]." and "We train diffusion models using the EDM configuration [4] with a fixed channel size of 128 on datasets of varying sizes [68, 137, 1094, 8750, 35000, 70000] until FID convergence. |