Fast Sampling of Diffusion Models via Operator Learning
Authors: Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Azizzadenesheli, Anima Anandkumar
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show our method achieves state-of-the-art FID of 3.78 for CIFAR-10 and 7.83 for Image Net-64 in the one-model-evaluation setting. |
| Researcher Affiliation | Collaboration | 1Caltech 2NVIDIA. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/devzhk/DSNO-pytorch. |
| Open Datasets | Yes | We first randomly sample a training set of ODE trajectories using the pre-trained diffusion model to be distilled. ... We mainly use ℓ1 loss for the experiments on CIFAR10 and Image Net-64. |
| Dataset Splits | No | The paper mentions training data and uses FID for evaluation, but does not specify distinct training, validation, and test splits with explicit percentages or sample counts. It refers to 'training set of ODE trajectories' but does not detail how the data for the main FID evaluation is split beyond that. |
| Hardware Specification | Yes | We compare the cost of one single forward pass of both DSNO and the original backbone1 on a V100 in a standard AWS p3.2xlarge instance. |
| Software Dependencies | No | The paper mentions implementing the backbone in Pytorch and using the Adam optimizer, but it does not specify version numbers for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | We use a batch size of 256 for CIFAR-10 experiments, a batch size of 2048 for Image Net experiments, and a batch size of 128 by default in our ablation study. We use the same base learning rate of 0.0002, learning rate warmup schedule, and β1, β2 of Adam (Kingma & Ba, 2014) as used in the diffusion model training without tuning these hyperparameters. |