Sampling with Mirrored Stein Operators
Authors: Jiaxin Shi, Chang Liu, Lester Mackey
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Sec. 5, we demonstrate the advantages of our algorithms on benchmark simplex-constrained problems from the literature, constrained sampling problems in post-selection inference (Taylor & Tibshirani, 2015; Lee et al., 2016; Tian et al., 2016), and unconstrained large-scale posterior inference with the Fisher information metric. Finally, we analyze the convergence of our mirrored algorithms in Sec. 6 and discuss our results in Sec. 7. 5 EXPERIMENTS We next conduct a series of simulated and real-data experiments to assess (1) distributional approximation on the simplex, (2) frequentist confidence interval construction for (constrained) post-selection inference, and (3) large-scale posterior inference with non-Euclidean geometry. |
| Researcher Affiliation | Industry | Jiaxin Shi1 Chang Liu2 Lester Mackey1 1 Microsoft Research New England 2 Microsoft Research Asia {jiaxinshi,chang.liu,lmackey}@microsoft.com |
| Pseudocode | Yes | Algorithm 1 Mirrored Stein Variational Gradient Descent & Stein Variational Mirror Descent |
| Open Source Code | Yes | https://github.com/thjashin/mirror-stein-samplers for Python and R code replicating all experiments. |
| Open Datasets | Yes | We first measure distributional approximation quality using two 20-dimensional simplex-constrained targets: the sparse Dirchlet posterior of Patterson & Teh (2013) extended to 20 dimensions and the quadratic simplex target of Ahn & Chewi (2020). |
| Dataset Splits | Yes | For each run, we randomly keep 20% of the dataset as test data, 20% of the remaining points as the validation set, and all the rest as the training set. |
| Hardware Specification | Yes | Results were recorded on an Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz and an NVIDIA Tesla P100 PCIe 16GB. |
| Software Dependencies | No | The paper mentions 'Python and R code' in the reproducibility statement, but does not specify version numbers for any software libraries, frameworks, or specific solvers. |
| Experiment Setup | Yes | The bandwidth ℓis determined by the median heuristic (Garreau et al., 2017). We select τ from {0.98, 0.99} for all SVMD experiments. For unconstrained targets, we report, for each method, results from the best fixed step size ϵ {0.01, 0.05, 0.1, 0.5, 1} selected on a separate validation set. For constrained targets, we select step sizes adaptively to accommodate rapid density growth near the boundary; specifically, we use RMSProp (Hinton et al., 2012)... |