GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework
Authors: Fangyikang Wang, Huminhao Zhu, Chao Zhang, Hanbin Zhao, Hui Qian
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on both synthetic and real-world data demonstrate the faster convergence and reduced approximation error of GAD-PVI methods over the state-of-the-art. We evaluate our algorithms on various synthetic and real-world tasks. The empirical results demonstrate the superiority of our GAD-PVI methods. |
| Researcher Affiliation | Academia | 1College of Computer Science and Technology, Zhejiang University 2Advanced Technology Institute, Zhejiang University 3State Key Lab of CAD&CG, Zhejiang University |
| Pseudocode | Yes | Algorithm 1: General Accelerated Dynamic-weight Particlebased Variational Inference (GAD-PVI) framework |
| Open Source Code | Yes | Our appendix can be downloaded at https://github.com/zituitui/GAD-PVI paper |
| Open Datasets | Yes | We follow the experiment setting in (Chen et al. 2018b), and use the dataset LIDAR which consists of 221 observations. In this experiment, we study a Bayesian regression task with Bayesian neural network on 4 datasets from UCI and LIBSVM. |
| Dataset Splits | Yes | In the BNN task, we split 1/5 of the training set as our validation set to tune the parameters. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions 'POT library (Flamary et al. 2021)' but does not specify a version number. Other software mentions are generic without version details. |
| Experiment Setup | Yes | For all the algorithms, the particles weights are initialized to be equal. In the BNN task, we split 1/5 of the training set as our validation set to tune the parameters. The position step-size are tuned via grid search for the fixed-weight Par VI algorithms, then used in the corresponding dynamic-weight algorithms. The acceleration parameters and weight adjustment parameters are tuned via grid search for each specific algorithm. We use a one-hidden-layer neural network with 50 hidden units and maintain 128 particles. For all the datasets, we set the batchsize as 128. |