Analysis of Variational Bayesian Latent Dirichlet Allocation: Weaker Sparsity Than MAP
Authors: Shinichi Nakajima, Issei Sato, Masashi Sugiyama, Kazuho Watanabe, Hiroko Kobayashi
NeurIPS 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | we theoretically investigate variational Bayesian (VB) learning in LDA. More specifically, we analytically derive the leading term of the VB free energy under an asymptotic setup, and show that there exist transition thresholds in Dirichlet hyperparameters around which the sparsity-inducing behavior drastically changes. Then we further theoretically reveal the notable phenomenon that VB tends to induce weaker sparsity than MAP in the LDA model, which is opposed to other models. We experimentally demonstrate the practical validity of our asymptotic theory on real-world Last.FM music data. |
| Researcher Affiliation | Collaboration | Shinichi Nakajima Berlin Big Data Center, TU Berlin Issei Sato University of Tokyo Tokyo Masashi Sugiyama University of Tokyo Tokyo Kazuho Watanabe Toyohashi University of Technology Aichi Hiroko Kobayashi Nikon Corporation Kanagawa |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any concrete access to source code, such as a specific repository link or an explicit code release statement. |
| Open Datasets | Yes | As a real-world dataset, we used the Last.FM dataset.4... 4http://mtg.upf.edu/node/1671 |
| Dataset Splits | No | The paper describes how artificial data was generated and how Last.FM data was selected ('randomly chose L artists... and M users'), but it does not provide specific dataset split information (exact percentages, sample counts, or detailed splitting methodology) for training, validation, or testing. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details, such as library or solver names with version numbers, needed to replicate the experiment. |
| Experiment Setup | No | The paper mentions data generation parameters and a high-level 'split and merge strategy' for finding solutions, but it does not provide specific experimental setup details such as concrete hyperparameter values (e.g., learning rate, batch size, number of epochs) or training configurations. |