Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Scalable Bayes via Barycenter in Wasserstein Space
Authors: Sanvesh Srivastava, Cheng Li, David B. Dunson
JMLR 2018 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that the geometric method approximates the full data posterior distribution better than its competitors across diverse simulations and reproduces known results when applied to a movie ratings database. [...] 4. Experiments We compared WASP with consensus Monte Carlo (CMC) (Scott et al., 2016), semiparametric density product (SDP) (Neiswanger et al., 2014), and variational Bayes (VB). |
| Researcher Affiliation | Academia | Sanvesh Srivastava EMAIL Department of Statistics and Actuarial Science University of Iowa Iowa City, Iowa 52242, USA Cheng Li EMAIL Department of Statistics and Applied Probability National University of Singapore Singapore 117546, Singapore David B. Dunson EMAIL Departments of Statistical Science, Mathematics, and ECE Duke University Durham, North Carolina 27708, USA |
| Pseudocode | Yes | Algorithm 1 Estimation of the WASP for f(θ) given samples of θ from k subset posteriors Input: Samples from k subset posteriors, {θji : θji Πm( | Y[j]), i = 1, . . . , sj, j = 1, . . . , k}; mesh size ϵ > 0. |
| Open Source Code | Yes | The code used in the experiments is available at https://github.com/blayes/WASP. |
| Open Datasets | Yes | 4.5 Real Data: Movie Lens Ratings Data We used Movie Lens data to illustrate the application of WASP to large-scale ratings data. Movie Lens data are one of the largest publicly available ratings data with about 10 million ratings from about 72 thousand users of the Movie Lens recommender system. |
| Dataset Splits | Yes | Let k be the number of subsets. The default strategy is to randomly allocate samples to subsets. [...] We randomly split the users into 10 training data sets such that ratings for any user belonged to the same training data set. To compute the approximate posteriors using CMC, SDP, and WASP, we set k = 10 and randomly partitioned the users into k subsets such that each subset contained all the ratings for a user. |
| Hardware Specification | Yes | All experiments ran on an Oracle Grid Engine cluster with 2.6GHz 16 core compute nodes. Full data posterior computations were allotted memory resources of 64GB, and all other methods were allotted memory resources of 16GB. |
| Software Dependencies | Yes | The sampling algorithm for the full data posterior was modified to obtain samples from the subset posteriors in CMC, SDP, and WASP. The sampling algorithms for subset posteriors in CMC and SDP were the same and were based on Equation (2) in Scott et al. (2016). The sampling algorithm for subset posteriors in WASP was based on (5). [...] The full data posterior distribution obtained using MCMC served as the benchmark in all our comparisons. Let π(θ | Y (n)) be the density of the full data posterior distribution for θ estimated using sampling and ˆπ(θ | Y (n)) be the density of an approximate posterior distribution for θ estimated using the WASP or its competitors. We used the following metric based on the total variation distance to compare the accuracy ˆπ(θ | Y (n)) in approximating π(θ | Y (n)) accuracy n ˆπ(θ | Y (n)) o = 1 1 ˆπ(θ | Y (n)) π(θ | Y (n)) d θ. (14) The accuracy metric lies in [0, 1] (Faes et al., 2012). The approximation of full data posterior density by ˆπ is poor or excellent if the accuracy metric is close to 0 or 1, respectively. In our experiments, we computed the kernel density estimates of ˆπ and π from the posterior samples of θ using R package Kern Smooth (Wand, 2015) and calculated the integral in (14) using numerical approximation. [...] A simple and efficient algorithm to find the WASP of a given function of parameters is summarized in Algorithm 1. [...] This linear program can be solved using a variety of linear programming solvers in Matlab or R, including the algorithms of Cuturi and Doucet (2014) and Srivastava et al. (2015). [...] by using Gurobi (Gurobi Optimization Inc., 2014). |
| Experiment Setup | Yes | Every sampling algorithm ran for 10,000 iterations. We discarded the first 5,000 samples as burn-in and thinned the chain by collecting every fifth sample. Convergence of the chains to their stationary distributions was confirmed using trace plots. [...] We chose two values of (p, r) {(4, 3), (80, 6)}, fixed n and s to be 6000 and 100,000, and randomly assigned the s observations to n samples. [...] Two values of k {10, 20} were used for CMC, SDP, and WASP, and the n samples were randomly partitioned into k subsets. [...] We compared Stan s HMC and SGLD with batch sizes 2000, 4000, step sizes 10 4, 10 5 and 104 iterations. |