DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics
Authors: Kaiwen Zheng, Cheng Lu, Jianfei Chen, Jun Zhu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that DPM-Solver-v3 achieves consistently better or comparable performance in both unconditional and conditional sampling with both pixelspace and latent-space DPMs, especially in 5 10 NFEs. We achieve FIDs of 12.21 (5 NFE), 2.51 (10 NFE) on unconditional CIFAR10, and MSE of 0.55 (5 NFE, 7.5 guidance scale) on Stable Diffusion, bringing a speed-up of 15% 30% compared to previous state-of-the-art training-free methods. |
| Researcher Affiliation | Collaboration | Kaiwen Zheng 1, Cheng Lu 1, Jianfei Chen1, Jun Zhu 123 1Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center, THBI Lab 1Tsinghua-Bosch Joint ML Center, Tsinghua University, Beijing, China 2Shengshu Technology, Beijing 3Pazhou Lab (Huangpu), Guangzhou, China |
| Pseudocode | Yes | We provide the pseudocode of the local approximation and global solver in Algorithm 1 and Algorithm 2, which concisely describes how we implement DPM-Solver-v3. [...] Algorithm 3 (n + 1)-th order singlestep solver |
| Open Source Code | Yes | Code is available at https://github.com/thu-ml/DPM-Solver-v3. |
| Open Datasets | Yes | We conduct extensive experiments on diverse image datasets... [24] (CIFAR10), [55] (LSUN-Bedroom), [9] (Image Net-256), [26] (MS-COCO2014) are cited as datasets used. |
| Dataset Splits | No | The paper mentions drawing K datapoints for EMS estimation and evaluating FID/MSE on 50k or 10k samples, but does not explicitly state the train/validation/test splits of the datasets used for model training or evaluation in terms of percentages or counts. |
| Hardware Specification | Yes | The total time for EMS computing is 7h on 8 GPU cards of NVIDIA A40. ... Table 4 shows the runtime of DPM-Solver-v3 and some other solvers on a single NVIDIA A40 under different settings. |
| Software Dependencies | No | We utilize the forward-mode automatic differentiation (torch.autograd.forward_ad) provided by Py Torch [39] to compute the Jacobian-vector-products (JVPs). No specific version numbers for PyTorch or other software dependencies are provided. |
| Experiment Setup | Yes | EMS computing We estimate the EMS at N = 1200 uniform timesteps λj0, λj1, . . . , λj N by drawing K = 4096 datapoints xλ0 q0... We use start time ϵ = 10 3 (NFE 10) and ϵ = 10 4 (NFE>10), end time T = 1 and adopt the uniform log SNR timestep schedule. For DPM-Solver-v3, we use the 3rd-order predictor with the 3rd-order corrector by default. |