Bayesian Deep Embedding Topic Meta-Learner
Authors: Zhibin Duan, Yishi Xu, Jianqiao Sun, Bo Chen, Wenchao Chen, Chaojie Wang, Mingyuan Zhou
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We apply the proposed framework to a hierarchical embedded topic model and achieve better performance than various baseline models on diverse experiments, including few-shot topic discovery and few-shot document classification. |
| Researcher Affiliation | Academia | 1National Laboratory of Radar Signal Processing, Xidian University, Xi an, China. 2Mc Combs School of Business, The University of Texas at Austin, Austin, TX 78712, USA. |
| Pseudocode | Yes | Algorithm 1 Autoencoding Variational Inference for Meta DETM Set mini-batch size m and the number of layer T Initialize the variational network parameters Ω; while not converge do 1. Randomly sample a mini-batch of m support and query sets to form a subtask n D(S) i om n D(Q) i om i=1; 2. Infer variational posterior for task latent variables {α(l) i }m,L i=1,l=1 and {c(l) i }m,L i=1,l=1 only using n D(S) i o 1,m by Eq. (5) and Eq. (8); 3. Infer variational posterior for document latent variables {θ(l) i,j}m,L,N i=1,l=1,j=1 using {Di,j}m,N i=1,j=1 by Eq. (3); 4. Calculate ΩL Ω; n D(S) i om according to Eq. (11) and update Ω. end while |
| Open Source Code | No | The paper does not provide an explicit statement or link for the release of open-source code for the described methodology. |
| Open Datasets | Yes | Our experiments are conducted on three widely-used textual benchmarks with different scales and document lengths, including 20Newsgroups (20NG) (Lang, 1995), Yahoo! Answers (Yahoo) (Zhang et al., 2015), and Reuters Corpus Volume I (RCV1) (Lewis et al., 2004). |
| Dataset Splits | Yes | For 20NG, we select 18 classes as the training corpora and the remaining two classes as the validation corpus and the test corpus, respectively; For Yahoo, 8 classes are chosen to form the training corpora, another two classes are used as the validation corpus and the test corpus, respectively. For RCV1, we split the total 55 classes into 47, 3, 5 to comprise the training, validation, and test corpus. |
| Hardware Specification | No | The paper mentions running experiments but does not specify any hardware details such as GPU models, CPU types, or memory. |
| Software Dependencies | No | The paper mentions the "Adam optimizer (Kingma & Ba, 2014)" but does not specify version numbers for any software dependencies or libraries (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | For hierarchical topic models, we set the network structure with three layers as [256, 128, 64]. For embedded topic models such as ETM, DETM, NS-DETM, HNS-DETM, and Meta-DETM, we set the embedding size as 50. For the NTMs, we set the hidden size as 256. For optimization, the Adam optimizer (Kingma & Ba, 2014) is adopted with an initial learning rate of 1e-2. We set support size as [3,5,10], and query size as 15, mini-batch size is defined as the class number of the training corpus. |