Reverse Diffusion Monte Carlo
Authors: Xunpeng Huang, Hanze Dong, Yifan HAO, Yian Ma, Tong Zhang
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our experiments, we find that a combination of the two approaches excels at distributions with multiple modes. For multi-modal target distributions such as those in Gaussian mixture models, rd MC greatly improves over the Langevin-style MCMC sampling methods both theoretically and in practice. In this section, we analyze the overall complexity of the RDMC via ULA inner loop estimation. Since the complexity of the importance sampling estimate is discussed in Section 3.3, we only consider Algorithm 1 with direct ULA sampling of q T t( |x) rather than the smart importance sampling initialization to make our analysis clear. F EMPIRICAL RESULTS |
| Researcher Affiliation | Collaboration | Xunpeng Huang Hanze Dong Yifan Hao Yi-An Ma Tong Zhang The Hong Kong University of Science and Technology Salesforce AI Research University of California, San Diego University of Illinois Urbana-Champaign |
| Pseudocode | Yes | Algorithm 1 RDMC: reverse diffusion Monte Carlo, Algorithm 2 ULA inner-loop for the qt( |x) sampler (Step 4 of Algorithm 1), Algorithm 3 Initialization of ˆp if ˆp = p |
| Open Source Code | No | The paper does not contain any explicit statements about releasing source code for the methodology or provide links to a code repository. |
| Open Datasets | No | The paper uses examples such as 'Gaussian mixture models', 'Potentials with Sub-Linear Tails' defined by equations, and 'Neal’s Funnel' which are mathematical distributions or synthetic problems, not external datasets for which access information would be provided. |
| Dataset Splits | No | The paper discusses concepts related to 'train' and 'validation' data in the context of diffusion models generally (e.g., 'training a parameterized diffusion model'), but does not provide specific train/validation/test splits for the experiments conducted in this paper. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, memory, or cloud computing instance types used for running the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., programming languages, libraries, or frameworks) used to replicate the experiments. |
| Experiment Setup | Yes | We choose 1, 000 particles in the experiments and use MMD (with RBF kernel) as the metric. We choose T { ln 0.99, ln 0.95, ln 0.9, ln 0.8, ln 0.7}. We use 10, 50, or 100 iterations to approximate ˆp chosen by the corresponding problem. The inner loop is initialized with importance sampling mean estimator by 100 particles. The inner iteration and inner loop sample-size are chosen from {1, 5, 10, 100}. The outer learning rate is chosen from {T/20, T/10, T/5}. When the algorithm converges, we further perform LMC until the limit of gradient complexity. |