Universal Boosting Variational Inference
Authors: Trevor Campbell, Xinglong Li
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on multiple datasets and models show that UBVI provides reliable, accurate posterior approximations. |
| Researcher Affiliation | Academia | Trevor Campbell Department of Statistics University of British Columbia Vancouver, BC V6T 1Z4 trevor@stat.ubc.ca Xinglong Li Department of Statistics University of British Columbia Vancouver, BC V6T 1Z4 xinglong.li@stat.ubc.ca |
| Pseudocode | Yes | Algorithm 1 The universal boosting variational inference (UBVI) algorithm. |
| Open Source Code | Yes | Code is available at www.github.com/trevorcampbell/ubvi. |
| Open Datasets | Yes | Real datasets available online at http://komarix.org/ac/ds/ and https://www.csie.ntu.edu.cn/~cjlin/libsvmtools/datasets/binary.html. |
| Dataset Splits | No | The paper mentions using a 'synthetic dataset', 'chemical reactivity dataset', and 'phishing websites dataset' with '20 subsampled datapoints' for the logistic regression, but it does not specify explicit training, validation, or test set splits or refer to standard predefined splits. |
| Hardware Specification | Yes | Each experiment was run 20 times with an Intel i7 8700K processor and 32GB of memory. |
| Software Dependencies | No | The paper mentions 'ADAM [55]' for optimization and the 'multivariate Gaussian family' for component distributions, but does not provide specific version numbers for any software libraries, programming languages, or frameworks used for implementation. |
| Experiment Setup | Yes | For all experiments, we used a regularization schedule of rn = 1/ n for BVI(+) in Eq. (1). We used the multivariate Gaussian family for H parametrized by mean and log-transformed diagonal covariance matrix. We used 10,000 iterations of ADAM [55] for optimization, with decaying step size 1/ 1 + i and Monte Carlo gradients based on 1,000 samples. Fully-corrective weight optimization was conducted via simplex-projected SGD for BVI(+) and nonnegative least squares for UBVI. Monte Carlo estimates of f, gn in UBVI were based on 10,000 samples. Each component optimization was initialized from the best of 10,000 trials of sampling a component (with mean m and covariance Σ) from the current mixture, sampling the initialized component mean from N(m, 16Σ), and setting the initialized component covariance to exp(Z)Σ, Z N(0, 1). |