Generalized Exponential Concentration Inequality for Renyi Divergence Estimation
Authors: Shashank Singh, Barnabas Poczos
ICML 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The main contribution of our work is to provide such a bound for an estimator of R enyi-α divergence for a smooth H older class of densities on the d-dimensional unit cube [0, 1]d. We also illustrate our theoretical results with a numerical experiment. |
| Researcher Affiliation | Academia | Shashank Singh SSS1@ANDREW.CMU.EDU Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15213 USA Barnab as P oczos BAPOCZOS@CS.CMU.EDU Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15213 USA |
| Pseudocode | No | The paper describes the estimation method mathematically but does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any information or links regarding the availability of source code for the described methodology. |
| Open Datasets | No | We used our estimator to estimate the R enyi α-divergence between two normal distributions in R3 restricted to the unit cube. In particular, for p = N( µ1, Σ), q = N( µ2, Σ). For each n ∈ {1, 2, 5, 10, 50, 100, 500, 1000, 2000, 5000}, n data points were sampled according to each distribution and constrained (via rejection sampling) to lie within [0, 1]3. The paper generates synthetic data and does not use or provide access to a pre-existing public dataset. |
| Dataset Splits | No | For each n ∈ {1, 2, 5, 10, 50, 100, 500, 1000, 2000, 5000}, n data points were sampled according to each distribution and constrained (via rejection sampling) to lie within [0, 1]3. Our estimator was computed from these samples, for α = 0.8, using the Epanechnikov Kernel K(u) = 3 4(1 u2) on [ 1, 1], with constant bandwidth h = 0.25. The true α-divergence was computed directly according to its definition on the (renormalized) distributions on [0, 1]3. The bias and variance of our estimator were then computed in the usual manner based on 100 trials. The paper describes a simulation setup and data generation, but not explicit train/validation/test splits from a dataset. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies or versions (e.g., libraries with version numbers) used for the experiments. |
| Experiment Setup | Yes | Our estimator was computed from these samples, for α = 0.8, using the Epanechnikov Kernel K(u) = 3 4(1 u2) on [ 1, 1], with constant bandwidth h = 0.25. |