Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm
Authors: Eli Sennesh, Hao Wu, Tommaso Salvatori
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, DCPC achieves better numerical performance than competing algorithms and provides accurate inference in a number of problems not previously addressed with predictive coding. We provide an open implementation of DCPC in Pyro on Github. Section 5 demonstrates DCPC experimentally in head-to-head comparisons against recent generative models and inference algorithms from the predictive coding literature. |
| Researcher Affiliation | Collaboration | Eli Sennesh1, Hao Wu2, Tommaso Salvatori2,3 1Department of Psychology, Vanderbilt University, Nashville, TN, USA 2VERSES AI Research Lab, Los Angeles, USA 3Vienna University of Technology, Vienna, Austria |
| Pseudocode | Yes | Algorithm 1 Divide-and-Conquer Predictive Coding for empirical Bayes |
| Open Source Code | Yes | We provide an open implementation of DCPC in Pyro on Github. |
| Open Datasets | Yes | We tested DCPC’s performance on elementary reconstruction and generation tasks by using it to train this exact generative model... Table 2: Negative log-likelihood and mean squared error for MCPC against DCPC on held-out images from the MNISTs. Means and standard deviations are taken across five random seeds. All experiments used a learning rate η = 0.1 and K = 4 particles. Figure 3 shows an extension of this experiment to EMNIST [Cohen et al., 2017] and Fashion MNIST [Xiao et al., 2017] as well as the original MNIST... Table 3: FID score comparisons on the Celeb A dataset [Liu et al., 2015]. |
| Dataset Splits | Yes | Figure 3: Top: images from validation sets of MNIST (left), EMNIST (middle), and Fashion MNIST (right). Bottom: reconstructions by deep latent Gaussian models trained with DCPC for MNIST (left), EMNIST (middle), and Fashion MNIST (right), averaging over K = 4 particles. DCPC achieves quality reconstructions by inference over z without training an inference network. The ground-truth images come from a 10% validation split of each data-set, on which DCPC only infers particles q K=4(z). |
| Hardware Specification | Yes | The initial DLGM experiments on the MNIST-alike datasets were performed on a desktop workstation with 128GB of RAM and an NVIDIA Quadro P4000 with 8GB of VRAM. Experiments on Celeb A were conducted on an NVIDIA DGX equipped with eight (8) NVIDIA A100 s, each with 80GB of VRAM. |
| Software Dependencies | No | The paper mentions implementing DCPC in 'Pyro' and optimizing with 'Adam', but it does not provide specific version numbers for Pyro or any other software libraries or frameworks used in the experiments. |
| Experiment Setup | Yes | All experiments used a learning rate η = 0.1 and K = 4 particles. We optimized the free energy in all experiments using Adam [Kingma and Ba, 2014]... Since Algorithm 1 updates θ only in its outer loop, we set S = 30 and ran DCPC for 150 epochs, with η = 0.001 and K = 16. |