Stochastic Localization via Iterative Posterior Sampling
Authors: Louis Grenioux, Maxence Noble, Marylou Gabrié, Alain Oliviero Durmus
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide numerical evidence of the robustness of the proposed approach in large dimension, beyond the class of distributions amenable to theoretical guarantees. Those results show that the proposed algorithm is on par or superior to modern sampling methods in a wide variety of settings. |
| Researcher Affiliation | Academia | 1CMAP, CNRS, Ecole polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France. |
| Pseudocode | Yes | Algorithm 1 SLIPS. Algorithm 2 Langevin-within-Langevin initialization. |
| Open Source Code | Yes | The code to reproduce our experiments is available at https://github.com/h2o64/slips. |
| Open Datasets | Yes | We consider in this paper the problem of sampling from a probability density known up to a normalization constant. This problem finds its origin in various tasks, ranging from Bayesian statistics (Kroese et al., 2011) to statistical mechanics (Krauth, 2006), including now generative modeling (Turner et al., 2019; Grenioux et al., 2023a). ...Bayesian Logistic Regression. Beyond toy distributions, we sample from the posterior of a Bayesian logistic regression model on two popular datasets : Ionosphere (d = 34) and Sonar (d = 61). ...High-dimensional Gaussian Mixtures. As a challenging task, we seek to estimate the relative weight of a bimodal Gaussian mixture with modes N (x; (2/3)1d, Σ) and N (x; (4/3)1d, Σ)... Field system ϕ4 from statistical mechanics. Lastly, we sample the 1D ϕ4 model, which was recently used as a benchmark in (Gabri e et al., 2022). |
| Dataset Splits | No | The paper mentions training and test datasets but does not specify a separate validation split or how validation was performed for its own experiments. |
| Hardware Specification | Yes | The computations were run on the same Nvidia V100 GPU. |
| Software Dependencies | No | The paper mentions using the 'POT library (Flamary et al., 2021)' but does not specify its version or any other software dependencies with version numbers. |
| Experiment Setup | Yes | For SLIPS, we consider three different SL schemes : Standard, Geom(1,1) and Geom(2,1). Except for RDMC, all the algorithms are informed by the scalar variance R2 π of the target distribution (or an estimation). We tuned the hyperparameters of each algorithm with coarse grid searches of similar size and similar computational budgets assuming access to an oracle distance metric to the target distribution (see details in Appendix H). ...Table 3: Hyper-parameter grids used for SLIPS on different targets. ...Table 4: Hyper-parameters selected for the experiments for each algorithm and target density. |