Convergence for score-based generative modeling with polynomial complexity
Authors: Holden Lee, Jianfeng Lu, Yixin Tan
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We prove the first polynomial convergence guarantees for the core mechanic behind SGM: drawing samples from a probability density p given a score estimate (an estimate of ln p) that is accurate in L2(p). Compared to previous works, we do not incur error that grows exponentially in time or that suffers from a curse of dimensionality. Our guarantee works for any smooth distribution and depends polynomially on its log-Sobolev constant. Using our guarantee, we give a theoretical analysis of score-based generative modeling, which transforms white-noise input into samples from a learned data distribution given score estimates at different noise scales. Our analysis gives theoretical grounding to the observation that an annealed procedure is required in practice to generate good samples, as our proof depends essentially on using annealing to obtain a warm start at each step. Moreover, we show that a predictor-corrector algorithm gives better convergence than using either portion alone. |
| Researcher Affiliation | Academia | Holden Lee Department of Applied Mathematics and Statistics Johns Hopkins University hlee283@jhu.edu Jianfeng Lu Department of Mathematics Duke University jianfeng@math.duke.edu Yixin Tan Department of Mathematics Duke University yixin.tan@duke.edu |
| Pseudocode | Yes | Algorithm 1 Annealed Langevin dynamics with estimated score [SE19] Algorithm 2 Predictor-corrector method with estimated score [Son+20b] |
| Open Source Code | No | The paper does not provide any concrete access to source code for the methodology described. The checklist explicitly states N/A for questions regarding code and experimental reproduction. |
| Open Datasets | No | The paper is theoretical and does not describe experiments performed on specific datasets. It refers to 'data distribution Pdata' as a theoretical concept but does not provide access information for any publicly available or open datasets used in empirical training. |
| Dataset Splits | No | The paper does not conduct experiments and therefore does not provide specific dataset split information (exact percentages, sample counts, or detailed splitting methodology) for reproduction. |
| Hardware Specification | No | The paper does not conduct experiments and thus does not provide specific hardware details (exact GPU/CPU models, processor types, or memory amounts) used for running experiments. |
| Software Dependencies | No | The paper is theoretical and does not describe an implementation of its methods, thus it does not list specific ancillary software details with version numbers. |
| Experiment Setup | No | The paper does not describe any specific experimental setup details, such as concrete hyperparameter values, training configurations, or system-level settings, as it is a theoretical work without empirical evaluations. |