Efficient Integrators for Diffusion Generative Models
Authors: Kushagra Pandey, Maja Rudolph, Stephan Mandt
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Applied to Phase Space Langevin Diffusion [Pandey & Mandt, 2023] on CIFAR-10, our deterministic and stochastic samplers achieve FID scores of 2.11 and 2.36 in only 100 network function evaluations (NFE) as compared to 2.57 and 2.63 for the best-performing baselines, respectively. |
| Researcher Affiliation | Collaboration | Kushagra Pandey Department of Computer Science University of California, Irvine pandeyk1@uci.edu; Maja Rudolph Bosch Center for Artificial Intelligence Maja.Rudolph@us.bosch.com; Stephan Mandt Department of Computer Science University of California, Irvine mandt@uci.edu |
| Pseudocode | Yes | Algorithm 1 Conjugate Integrators (defined in Eqn. 8); Algorithm 2 Conjugate Symplectic Euler; Algorithm 3 Conjugate Velocity Verlet; Algorithm 4 Conjugate OBA |
| Open Source Code | Yes | Our code and model checkpoints will be made publicly available at https://github.com/mandt-lab/PSLD. |
| Open Datasets | Yes | We use the CIFAR-10 (Krizhevsky, 2009) (50k images), Celeb A-64 (downsampled to 64 x 64 resolution, 200k images) (Liu et al., 2015) and the AFHQv2-64 (Choi et al., 2020) (downsampled to 64 x 64 resolution, 15k images) datasets for comparisons. |
| Dataset Splits | No | The paper mentions using CIFAR-10, Celeb A-64, and AFHQv2-64 datasets but does not explicitly state specific train/validation/test splits (e.g., percentages or exact counts for each split). |
| Hardware Specification | Yes | Table 3: # GPUs 8 |
| Software Dependencies | Yes | The above matrix exponential can be computed using standard scientific libraries like Py Torch (Paszke et al., 2019) or Sci Py (Virtanen et al., 2020). ... For numerical integration, we use the odeint method from the torchdiffeq package (Chen, 2018). |
| Experiment Setup | Yes | For all ablation results in Section 3 in the main text, we use pre-trained PSLD (Pandey & Mandt, 2023) models for CIFAR-10 with SDE hyperparameters Γ = 0.01, ν = 4.01 and β = 8.0. ... Table 3 summarizes the different training hyperparameters across datasets. We use the Hybrid Score Matching (HSM) objective during training. ... We use quadratic striding for timestep discretization proposed in Dockhorn et al. (2022b) during sampling, which ensures more number of score function... Similar to PSLD, we set ϵ = 1e 3 during sampling for all experiments. |