On Convergence of Polynomial Approximations to the Gaussian Mixture Entropy
Authors: Caleb Dahlke, Jason Pacheco
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental validation supports our theoretical results while showing that our method is comparable in computation to Huber et al. We conclude with an empirical comparison of all polynomial approximations that produce the divergent behavior of the Huber et al. approximation while our propsed methods maintain convergence. We also compare accuracy and compuation time for each method accross a varaitey of dimensions, number of GMM components, and polynomial orders. Finally, we show an application of our methods in Nonparametric Variational Inference [11] where the guarantees of convergence play a large role in the accuracy of posterior approximation via GMMs. |
| Researcher Affiliation | Academia | Caleb Dahlke Department of Mathematics University of Arizona Jason Pacheco Department of Computer Science University of Arizona |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statements about open-source code release or links to code repositories. |
| Open Datasets | No | The paper states, 'To highlight the theoretical properties, such as convergence, divergence, accuracy, and lower-bound of methods as discussed in Sec. 4, we will consider some synthetic GMMs.' and 'In each experiment, we are approximating a multivariate mixture T distribution, p(x). We randomize the parameters of p(x) and the initialization parameters of the variational GMM, q(x), for optimization.' This indicates the use of synthetic data or data generated for the experiments, but no concrete access information (link, citation, repository) to a publicly available or open dataset is provided. |
| Dataset Splits | No | The paper describes generating synthetic data and randomizing parameters for optimization but does not provide specific training, validation, or test dataset splits (e.g., percentages, sample counts, or citations to standard splits). |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments, such as GPU/CPU models or memory specifications. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers needed to replicate the experiments. |
| Experiment Setup | Yes | We use a simple two-component GMM with parameters w1 = 0.35, w2 = 0.65, µ1 = 2, µ2 = 1, σ2 1 = 2, and σ2 2 (0, 1]. We are changing the variance of the second Gaussian, σ2 2, in the range (0, 1]. ... We create two GMMs similar to the example published in [15]... We consider a five-component, two-dimensional GMM with the parameters wi = .2 i, µ1 = [0, 0]T , µ2 = [3, 2]T , µ3 = [1, .5]T , µ4 = [2.5, 1.5]T , µ5 = c[1, 1]T for c [ 3, 3], Σ1 = .25I2, Σ2 = 3I2, and Σ3 = Σ4 = Σ5 = 2I2 where I2 is the two dimensional identity matrix. ... The KL is approximated using a 100000 Monte Carlo approximation after convergence of each algorithm. ... The authors iterate between optimizing the mean components, µi, using L1(q) and optimizing the variance components, σ2 i , using L2(q) until the second approximate appropriately converges δL2(q) < .0001. |