A Heavy-Tailed Algebra for Probabilistic Programming
Authors: Feynman T. Liang, Liam Hodgkinson, Michael W. Mahoney
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical results confirm that inference algorithms that leverage our heavy-tailed algebra attain superior performance across a number of density modeling and variational inference (VI) tasks. 4 Experiments We now demonstrate that GGA-based density estimation yields improvements in tail estimation across several metrics. |
| Researcher Affiliation | Academia | Feynman Liang Department of Statistics University of California, Berkeley feynman@berkeley.edu Liam Hodgkinson School of Mathematics and Statistics University of Melbourne, Australia lhodgkinson@unimelb.edu.au Michael W. Mahoney ICSI, LBNL, and Department of Statistics University of California, Berkeley mmahoney@stat.berkeley.edu |
| Pseudocode | Yes | Algorithm 1: GGA tails static analysis pass Data: Abstract syntax tree for a PPL program Result: GGA parameter estimates for all random variables frontier [rv : Parents(rv) = ]; tails {}; while frontier = do next frontier.pop Left(); tails[next] compute GGA(next.op, next.parent); frontier frontier + next.children(); end return tails |
| Open Source Code | Yes | To illustrate an implementation of GGA for static analysis, we sketch the operation of the PPL compiler at a high-level and defer to the code in Supplementary Materials for details. |
| Open Datasets | Yes | super (superconductor critical temperature prediction dataset [23] with n = 256 and d = 154); who (life expectancy data from the World Health Organisation in the year 2013 [41] with n = 130, d = 18); air (air quality data [14] with n = 6941, d = 11); and blog (blog feedback prediction dataset [7] with n = 1024, d = 280). |
| Dataset Splits | No | All experiments are repeated for 100 trials, trained to convergence using the Adam optimizer with manually tuned learning rate. Additional details are available in Appendix D. |
| Hardware Specification | No | Our implementation target beanmachine [50] is a declarative PPL selected due to availability of a PPL compiler and support for static analysis plugins. Similar to [4, 46], it uses Py Torch [40] for GPU tensors and automatic differentiation. |
| Software Dependencies | No | To illustrate an implementation of GGA for static analysis, we sketch the operation of the PPL compiler at a high-level and defer to the code in Supplementary Materials for details. A probabilistic program is first inspected using Python s built-in ast module... it uses Py Torch [40] for GPU tensors and automatic differentiation. trained to convergence using the Adam optimizer with manually tuned learning rate. |
| Experiment Setup | No | All experiments are repeated for 100 trials, trained to convergence using the Adam optimizer with manually tuned learning rate. Additional details are available in Appendix D. |