Sampling in Unit Time with Kernel Fisher-Rao Flow
Authors: Aimee Maurais, Youssef Marzouk
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We introduce a new mean-field ODE and corresponding interacting particle systems (IPS) for sampling from an unnormalized target density. The IPS are gradient-free, available in closed form, and only require the ability to sample from a reference density and compute the (unnormalized) target-to-reference density ratio. The mean-field ODE is obtained by solving a Poisson equation for a velocity field that transports samples along the geometric mixture of the two densities, π1 t 0 πt 1, which is the path of a particular Fisher Rao gradient flow. We employ a RKHS ansatz for the velocity field, which makes the Poisson equation tractable and enables discretization of the resulting mean-field ODE over finite samples. The mean-field ODE can be additionally be derived from a discrete-time perspective as the limit of successive linearizations of the Monge Amp ere equations within a framework known as sampledriven optimal transport. We introduce a stochastic variant of our approach and demonstrate empirically that our IPS can produce high-quality samples from varied target distributions, outperforming comparable gradient-free particle systems and competitive with gradient-based alternatives. |
| Researcher Affiliation | Academia | 1Center for Computational Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA. |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. The methods are described using mathematical equations and textual explanations. |
| Open Source Code | Yes | Code for the experiments is available at at https://github.com/amaurais/KFRFlow.jl. |
| Open Datasets | No | The paper mentions "The reference distribution π0 is always standard Gaussian." and describes problem setups like "Two-Dimensional Bayesian Posteriors" with specific likelihood definitions, but it does not provide concrete access information (link, DOI, citation) for a publicly available dataset used for training or evaluation. The datasets are constructed based on specific likelihood functions rather than being pre-existing public datasets. |
| Dataset Splits | No | The paper does not explicitly provide training/test/validation dataset splits. It describes generating "target ensembles" for evaluation but does not detail how data is partitioned into these specific splits for reproduction purposes. |
| Hardware Specification | Yes | Benchmarks were performed in Julia using Benchmark Tools.jl (Chen & Revels, 2016) on a 2020 Mac Book Air with Apple M1 processor. |
| Software Dependencies | No | The paper states: "We perform all experiments in Julia using the package Differential Equations.jl (Rackauckas & Nie, 2017)". However, it does not provide specific version numbers for Julia or the DifferentialEquations.jl package, which are necessary for full reproducibility. |
| Experiment Setup | Yes | Table 2: Funnels: Selected hyperparameters for each algorithm and target dimension d. lambda (KFRFlow-I) 0.01 0.001 0.001 0.001 epsilon (KFRD) 5 5 5 2.5 lambda (KFRD) 0.001 0.01 0.1 0.1 T (CBS) 25 12.5 25 25 beta (CBS) 0.125 0.5 0.25 0.25 T (SVGD) 100 100 100 100 T (ULA) 12.5 12.5 25 12.5 |