Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Learned Reference-based Diffusion Sampler for multi-modal distributions
Authors: Maxence Noble, Louis Grenioux, Marylou Gabrié, Alain Oliviero Durmus
ICLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally demonstrate that LRDS best exploits prior knowledge on the target distribution compared to competing algorithms on a variety of challenging distributions. To validate our approach, we compare GMM-LRDS and EBM-LRDS on a variety of multi-modal distributions against the following annealed methods: (a) annealed MCMC methods Sequential Monte Carlo (SMC) and Replica Exchange (RE); (b) variational diffusion-based methods, implemented with the LV loss LV-PIS (Zhang & Chen, 2022), LV-DDS (Vargas et al., 2023a), LV-DIS (Berner et al., 2023) and LV-CMCD (Vargas et al., 2024); (c) adaptive diffusion-based approaches i DEM (Akhound-Sadegh et al., 2024) and PDDS (Phillips et al., 2024). |
| Researcher Affiliation | Academia | Maxence Noble , Louis Grenioux , Marylou Gabri e & Alain Oliviero Durmus CMAP, CNRS Ecole polytechnique, Institut Polytechnique de Paris 91120 Palaiseau, France. Corresponding authors: EMAIL |
| Pseudocode | Yes | A PSEUDO-CODES OF RDS-BASED ALGORITHMS We respectively give sampling procedures and training procedures of a general version of RDS in Algorithm 1 and Algorithm 2. Relying on this, we derive the complete training schemes for GMM-LRDS (Algorithm 3) and EBM-LRDS (Algorithm 4). |
| Open Source Code | Yes | Our codebase is available at https://github.com/h2o64/sde_sampler_lrds. |
| Open Datasets | Yes | Finally, we evaluate the performance of a Bayesian logistic model... Following Blessing et al. (2024), we consider four real-world settings of binary classification problem: Ionosphere (dim = 35, M = 351), Sonar (dim = 61, M = 208), German Credit (dim = 25, M = 1000), Breast Cancer (dim = 31, M = 569). Each of these datasets is randomly split into a training subset Dtrain of size 0.8M and a test subset Dtest of size 0.2M. |
| Dataset Splits | Yes | Each of these datasets is randomly split into a training subset Dtrain of size 0.8M and a test subset Dtest of size 0.2M. |
| Hardware Specification | Yes | The computations were all ran on the same V100 GPU. |
| Software Dependencies | No | For GMM-LRDS, the EM algorithm is taken from Pedregosa et al. (2011). - While this implies scikit-learn, it does not provide a specific version number for the library or any other software dependencies. |
| Experiment Setup | Yes | For all of them, we perform 4096 optimization steps with a batch of size 2048. The neural network at stake is a Fourier MLP, as in Zhang & Chen (2022), with 4 layers of width 64. In the case of PIS, DDS and DIS, we use a target-informed parameterization by adding NN(t) log π(x) (where NN is a time-dependent scalar neural network) to the Fourier MLP, as suggested by the respective authors. As recommended by Vargas et al. (2024), we do not consider this extra-parameterization in CMCD, since the drift of the generative process is already informed by π. We highlight that, by default, we do not use this target-informed parameterization in LRDS, since the reference process is specifically designed to be it-self target-informed, hence avoiding useless evaluations of the target score. Additionally, as recommended by Zhang & Chen (2022), we design the LRDS guidance network such that gθ0 = 0. This ensures that the very first sampling phase is driven solely using the reference process. |