Score-Based Generative Modeling through Stochastic Differential Equations
Authors: Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Combined with multiple architectural improvements, we achieve record-breaking performance for unconditional image generation on CIFAR-10 with an Inception score of 9.89 and FID of 2.20, a competitive likelihood of 2.99 bits/dim, and demonstrate high fidelity generation of 1024 ˆ 1024 images for the first time from a score-based generative model. (Abstract) and Table 1: Comparing different reverse-time SDE solvers on CIFAR-10. (Section 4.1) |
| Researcher Affiliation | Collaboration | Yang Song Stanford University yangsong@cs.stanford.edu Jascha Sohl-Dickstein Google Brain jaschasd@google.com Diederik P. Kingma Google Brain durk@google.com Abhishek Kumar Google Brain abhishk@google.com Stefano Ermon Stanford University ermon@cs.stanford.edu Ben Poole Google Brain pooleb@google.com |
| Pseudocode | Yes | Please find pseudo-code and a complete description in Appendix G. (Section 4.2) and Algorithm 1 Predictor-Corrector (PC) sampling (Appendix G) |
| Open Source Code | Yes | Code and checkpoints are open-sourced at https://github.com/yang-song/score sde. (Appendix H) |
| Open Datasets | Yes | For VE SDEs, we consider two datasets: 32 ˆ 32 CIFAR-10 (Krizhevsky et al., 2009) and 64 ˆ 64 Celeb A (Liu et al., 2015), pre-processed following Song & Ermon (2020). (Appendix H.1) and 1024 ˆ 1024 Celeb A-HQ (Karras et al., 2018) (Appendix H.3) |
| Dataset Splits | No | The paper references datasets but does not explicitly provide specific percentages or absolute counts for training, validation, and test splits, nor does it cite a source for predefined splits. |
| Hardware Specification | No | The paper mentions 'TensorFlow Research Cloud' in acknowledgements, but does not provide specific hardware details (like exact GPU/CPU models, memory, or detailed computer specifications) used for running experiments. |
| Software Dependencies | No | The paper mentions 'tensorflow gan' and 'scipy.integrate.solve_ivp' but does not provide specific version numbers for these or any other key software components, making it non-reproducible in terms of software dependencies. |
| Experiment Setup | Yes | Unless otherwise noted, all models are trained for 1.3M iterations, and we save one checkpoint per 50k iterations. (Appendix H.1) and Unless otherwise noted, models are trained with the original discrete SMLD and DDPM objectives in Eqs. (1) and (3) and use a batch size of 128. (Appendix H.1) and The exponential moving average (EMA) rate has a significant impact on performance. For models trained with VE perturbations, we notice that 0.999 works better than 0.9999... We therefore use an EMA rate of 0.999 and 0.9999 for VE and VP models respectively. (Appendix H.1) |