Rethinking LDA: Moment Matching for Discrete ICA
Authors: Anastasia Podosinnikova, Francis Bach, Simon Lacoste-Julien
NeurIPS 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, (a) we compare experimentally the GP/DICA cumulants with the LDA moments and (b) the spectral algorithm [3], the tensor power method [4] (TPM), the joint diagonalization (JD) algorithm from Algorithm 1, and variational inference for LDA [1]. Real data: the associated press (AP) dataset, from D. Blei s web page,4 with N = 2, 243 documents and M = 10, 473 vocabulary words and the average document length b L = 194; the NIPS papers dataset5 [28] of 2, 483 NIPS papers and 14, 036 words, and b L = 1, 321; the KOS dataset,6 from the UCI Repository, with 3, 430 documents and 6, 906 words, and b L = 136. |
| Researcher Affiliation | Academia | Anastasia Podosinnikova Francis Bach Simon Lacoste-Julien INRIA Ecole normale sup erieure Paris |
| Pseudocode | Yes | Algorithm 1 Joint diagonalization (JD) algorithm for GP/DICA cumulants (or LDA moments) |
| Open Source Code | Yes | We use our Matlab implementation of the GP/DICA cumulants, the LDA moments, and the diagonalization algorithms. The datasets and the code for reproducing our experiments are available online.7 Footnote 7: https://github.com/anastasia-podosinnikova/dica |
| Open Datasets | Yes | Real data: the associated press (AP) dataset, from D. Blei s web page,4 with N = 2, 243 documents and M = 10, 473 vocabulary words and the average document length b L = 194; the NIPS papers dataset5 [28] of 2, 483 NIPS papers and 14, 036 words, and b L = 1, 321; the KOS dataset,6 from the UCI Repository, with 3, 430 documents and 6, 906 words, and b L = 136. Footnote 4: http://www.cs.columbia.edu/ blei/lda-c Footnote 5: http://ai.stanford.edu/ gal/data Footnote 6: https://archive.ics.uci.edu/ml/datasets/Bag+of+Words |
| Dataset Splits | No | The paper mentions 'held out documents' for evaluation but does not specify explicit training, validation, and test dataset splits with percentages or sample counts. It describes how semi-synthetic data is generated and evaluated, and how real data is evaluated using 'held out documents', but without precise split information. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models, memory specifications, or cloud computing resources used for running the experiments. It only mentions a 'Matlab implementation'. |
| Software Dependencies | No | The paper mentions using a 'Matlab implementation' but does not specify the version of Matlab or any other software dependencies with their respective version numbers. |
| Experiment Setup | Yes | All the sampling models have the parameter c which is set to c = c0 c/ c 1, where c is the learned c from the real dataset with variational LDA, and c0 is a parameter that we can vary. The GP data are sampled from the gamma-Poisson model (3) with b = c0/b L so that the expected document length is b L. For the AP data, K {10, 50} topics are learned and, for the NIPS data, K {10, 90} topics are learned. |