Near-Optimality of Contrastive Divergence Algorithms
Authors: Pierre Glaser, Kevin Han Huang, Arthur Gretton
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We perform a non-asymptotic analysis of the contrastive divergence (CD) algorithm, a training method for unnormalized models. While prior work has established that (for exponential family distributions) the CD iterates asymptotically converge at an O(n 1/3) rate to the true parameter of the data distribution, we show, under some regularity assumptions, that CD can achieve the parametric rate O(n 1/2). |
| Researcher Affiliation | Academia | Pierre Glaser Kevin Han Huang Arthur Gretton Gatsby Computational Neuroscience Unit, University College London pierreglaser@gmail.com, han.huang.20@ucl.ac.uk, arthur.gretton@gmail.com |
| Pseudocode | Yes | Algorithm 1 Online CD |
| Open Source Code | No | The paper does not include experiments, and thus no code or data. |
| Open Datasets | No | The paper is a theoretical analysis and does not involve empirical studies with specific datasets for training or evaluation. |
| Dataset Splits | No | The paper is a theoretical analysis and does not involve empirical studies with specific dataset splits for validation. |
| Hardware Specification | No | The paper is a theoretical analysis and does not describe any specific hardware used for experiments. |
| Software Dependencies | No | The paper is a theoretical analysis and does not list specific software dependencies with version numbers for experimental replication. |
| Experiment Setup | No | The paper is a theoretical analysis and does not describe an experimental setup, hyperparameters, or system-level training settings. |