Approximate Bayesian Inference with Stein Functional Variational Gradient Descent
Authors: Tobias Pielok, Bernd Bischl, David Rügamer
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show the efficacy of training BNNs with SFVGD on various real-world datasets. |
| Researcher Affiliation | Academia | Tobias Pielok, Bernd Bischl, David Rügamer Department of Statistics, LMU Munich, Munich, Germany Munich Center for Machine Learning, Munich, Germany {tobias.pielok, bernd.bischl, david.ruegamer}@stat.uni-muenchen.de |
| Pseudocode | Yes | Algorithm 1: Stein Functional Variational Gradient Descent Step sfvgd_step; Algorithm 2: Stein Functional Variational Neural Network; Algorithm 3: Stein Functional Variational Gradient Boosting |
| Open Source Code | No | The paper does not provide a specific repository link or an explicit statement about the release of their source code. |
| Open Datasets | Yes | We use the benchmark data setup proposed by Hernandez-Lobato & Adams (2015) and Sun et al. (2019) for evaluating probabilistic regression approaches. This setup includes selected data sets from the UCI repository namely, the four smaller data sets Concrete, Energy, Wine, Yacht, and the four larger data sets Naval, Protein, Video (Memory and Time), and GPU. In addition to Sun et al. (2019), we also compare our approaches on three additional smaller data sets (Airfoil, Diabetes, Forest Fire)... In Table 5, the data characteristics and pre-processing steps are listed. |
| Dataset Splits | Yes | All data sets are standardized prior to model fitting and split into 90% training data and 10% test data. For the comparisons, this splitting process is repeated 10 times based on 10 different splits to also evaluate the variability of each method. |
| Hardware Specification | Yes | All experiments and benchmarks were carried out on an internal cluster with Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz, 32 cores, 64 GB Random-access memory, and operating system Ubuntu 20.04.1 LTS. |
| Software Dependencies | No | The paper only mentions the operating system version ("Ubuntu 20.04.1 LTS") but does not list specific versions for other key software components or libraries (e.g., Python, PyTorch, TensorFlow). |
| Experiment Setup | Yes | All data sets are standardized prior to model fitting and split into 90% training data and 10% test data. For the comparisons, this splitting process is repeated 10 times based on 10 different splits to also evaluate the variability of each method. Details for each procedure are given in the Appendix. BNN approaches are fitted using the recommended architecture and tuning parameters by Sun et al. (2019). We reduced the epochs from 10,000 to 1,000 epochs for the smaller data sets to reduce the computational runtime. The hyperparameters of the SFVGD step are the same as the ones we used for SFVNN. |