PFGM++: Unlocking the Potential of Physics-Inspired Generative Models
Authors: Yilun Xu, Ziming Liu, Yonglong Tian, Shangyuan Tong, Max Tegmark, Tommi Jaakkola
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that models with finite D can be superior to previous stateof-the-art diffusion models on CIFAR-10/FFHQ 64 64 datasets/LSUN Churches 256 256, with median Ds. In class-conditional setting, D=2048 yields current state-of-the-art FID of 1.74 on CIFAR-10 without additional training. |
| Researcher Affiliation | Academia | 1Massachusetts Institute of Technology, MIT, Cambridge, MA, USA. |
| Pseudocode | Yes | Algorithm 1 EDM training Algorithm 2 PFGM++ training with hyperparameter transferred from EDM Algorithm 3 EDM sampling (Heun s 2nd order method) Algorithm 4 PFGM++ training with hyperparameter transferred from EDM Algorithm 5 DDPM training Algorithm 6 PFGM++ training with hyperparameter transferred from DDPM Algorithm 7 DDIM sampling Algorithm 8 PFGM++ sampling transferred from DDIM |
| Open Source Code | Yes | Code is available at https://github. com/Newbeeer/pfgmpp |
| Open Datasets | Yes | We consider the widely used benchmarks CIFAR-10 32 32 (Krizhevsky, 2009), FFHQ 64 64 (Karras et al., 2018) and LSUN Churches 256 256 (Yu et al., 2015) for image generation. |
| Dataset Splits | No | The paper mentions training, testing, and evaluation on datasets like CIFAR-10, FFHQ, and LSUN Churches. However, it does not explicitly state the specific training/validation/test splits (e.g., percentages or sample counts) used for these datasets in the provided text. |
| Hardware Specification | Yes | All the experiments are run on four NVIDIA A100 GPUs or eight NVIDIA V100 GPUs. |
| Software Dependencies | No | The paper mentions using specific architectures like 'NCSN++' and 'DDPM++', and the 'Adam optimizer', but it does not specify software dependencies with version numbers (e.g., PyTorch version, Python version, CUDA version). |
| Experiment Setup | Yes | For training, we utilize the improved NCSN++/DDPM++ architectures, preconditioning techniques and hyperparameters from the state-of-the-art diffusion model EDM (Karras et al., 2022). Specifically, we use the alignment method in section 4 to transfer their well-tuned hyper-parameters. For optimizers, following EDM, we adopt the Adam optimizer with a learning rate of 10e 4. We further incorporate the EMA schedule, learning rate warm-up, and data augmentations in EDM. |