SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence
Authors: Sinho Chewi, Thibaut Le Gouic, Chen Lu, Tyler Maunu, Philippe Rigollet
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experiments To implement Algorithm 1, we numerically approximate the kernel K = KL given in (11). When π is the standard Gaussian distribution on R, the eigendecomposition of the operator L in (9) is known explicitly in terms of the Hermite polynomials [BGL14, 2.7.1], and we approximate the kernel via a truncated sum: ˆK(x, y) = Pk i=1 λ 1 i φi(x)φi(y) (Figure 2) involving the smallest eigenvalues of L . In the general case, we implement a basic finite difference (FD) method to approximate the eigenvalues and eigenfunctions of L . |
| Researcher Affiliation | Academia | Sinho Chewi MIT schewi@mit.edu Thibaut Le Gouic MIT tlegouic@mit.edu Chen Lu MIT chenl819@mit.edu Tyler Maunu MIT maunut@mit.edu Philippe Rigollet MIT rigollet@mit.edu |
| Pseudocode | Yes | Algorithm 1 LAWGD 1: procedure LAWGD(KL , µ0) 2: draw N particles X[1] 0 , . . . , X[N] 0 i.i.d. µ0 3: for t = 1, . . . , T 1 do 4: for i = 1, . . . , N do 5: X[i] t+1 X[i] t h N PN j=1 1KL (X[i] t , X[j] t ) 6: end for 7: end for 8: return X[1] T , . . . , X[N] T 9: end procedure |
| Open Source Code | No | The paper does not provide any statement about making its source code publicly available or a link to a code repository. |
| Open Datasets | No | The paper uses a synthetic mixture of Gaussians as its target distribution for sampling experiments, and does not provide any access information (link, citation) for a publicly available dataset. |
| Dataset Splits | No | The paper describes sampling experiments but does not provide specific details on training, validation, or test dataset splits. |
| Hardware Specification | No | No specific hardware details (like CPU/GPU models, memory) are provided. The paper only mentions running on a grid of points and using finite difference methods, but not the specific computational resources. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used in the implementation. |
| Experiment Setup | Yes | We compare LAWGD with SVGD using the RBF kernel and median-based bandwidth as in [LW16]. We approximate the eigenfunctions and eigenvalues using a finite difference scheme, on 256 grid points evenly spaced between 14 and 14. Constant step sizes for LAWGD and SVGD are tuned and the algorithms are run for 5000 iterations, and the samples are initialized to be uniform on [1, 4]. The results are displayed in Figure 3. All 256 discrete eigenfunctions and eigenvalues are used. |