Stochastic Stein Discrepancies
Authors: Jackson Gorham, Anant Raj, Lester Mackey
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We complement these results with a series of experiments illustrating the application of SSDs to biased MCMC hyperparameter tuning, approximate MCMC sampler selection, and particle-based variational inference. In each case, we find that SSDs deliver inferences equivalent to or more accurate than standard SDs with orders of magnitude fewer datapoint accesses. |
| Researcher Affiliation | Collaboration | Jackson Gorham Whisper.ai, Inc jackson@whisper.ai Anant Raj MPI for Intelligent Systems Tübingen, Germany anant.raj@tuebingen.mpg.de Lester Mackey Microsoft Research New England lmackey@microsoft.com |
| Pseudocode | Yes | Algorithm 1 Stochastic Stein Variational Gradient Descent (SSVGD) |
| Open Source Code | Yes | Julia [6] code recreating the experiments in Sections 5.1 and 5.2 and Python code recreating the experiments in Section 5.3 is available at https://github.com/jgorham/stochastic_stein_discrepancy. |
| Open Datasets | Yes | We used the same model parameterization as Welling and Teh [50], which was a posterior distribution with L = 100 likelihood terms contributing to the posterior density. ... The training set was constructed by selecting a subset of 10, 000 images from the MNIST dataset that had a 7 or 9 label, and then reducing each covariate vector of 784 pixel values ... The boston dataset was first published in [24] while the latter two are available on the UCI repository [13]. |
| Dataset Splits | No | No specific dataset split percentages, counts, or explicit validation split information was found. The paper mentions train and test sets but not a distinct validation set split. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory specifications) used for running experiments were provided. |
| Software Dependencies | No | The paper mentions 'Julia [6] code' and 'Python code' but does not specify versions for these languages or any libraries used in the experiments. |
| Experiment Setup | Yes | A first step in using SGLD is selecting an appropriate step size ǫ... For each sample of size n, we computed the IMQ KSD without any subsampling and the SKSD with batch sizes m = 1 and m = 10. ... we compare SSVGD (Algorithm 1) with minibatch sizes m = 0.1L and m = 0.25L with standard SVGD (m = L). |