SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models
Authors: Shuchen Xue, Mingyang Yi, Weijian Luo, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, Zhi-Ming Ma
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that SA-Solver achieves: 1) improved or comparable performance compared with the existing state-of-the-art (SOTA) sampling methods for few-step sampling; 2) SOTA FID on substantial benchmark datasets under a suitable number of function evaluations (NFEs). |
| Researcher Affiliation | Collaboration | Shuchen Xue1,4 , Mingyang Yi2 , Weijian Luo3 , Shifeng Zhang2,Jiacheng Sun2, Zhenguo Li2, Zhi-Ming Ma1,4 1University of Chinese Academy of Sciences 2 Huawei Noah s Ark Lab 3 Peking University 4Academy of Mathematics and Systems Science, Chinese Academy of Sciences |
| Pseudocode | Yes | Algorithm 1 SA-Solver |
| Open Source Code | No | The paper does not contain an explicit statement or link for open-source code release. |
| Open Datasets | Yes | We test on Image Net 256x256 (latent diffusion model) with τ(t) 1. Results of the data-prediction and noise-prediction model are shown in Table 1. It can be seen that the data-prediction model can achieve better sampling quality values under different NFEs, thus we use the data-prediction model in the rest of the experiments. More detailed discussions and theoretical analysis can be seen in Appendix A.2.4. We conduct an ablation study on the CIFAR10 dataset as follows. We use EDM [27] baseline-VE pretrained checkpoint. |
| Dataset Splits | No | The paper mentions training on datasets but does not explicitly provide details about training/validation/test splits. |
| Hardware Specification | No | The paper does not provide specific details on the hardware used for experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies. |
| Experiment Setup | Yes | For ease of computation, we take τ(t) τ as a constant function or a piecewise constant function. We leave the detailed settings for τ(t), predictor step, and corrector step in Appendix E. For the following experiments, we first discuss the effectiveness of the data-prediction model. Then we evaluate the performance of SA-Solver under different random noise scales τ to demonstrate the principles for selecting τ under few-steps and a considerable number of steps. Finally, we compare SA-Solver with the existing solver to demonstrate its effectiveness. |