Discovering Discrete Latent Topics with Neural Variational Inference
Authors: Yishu Miao, Edward Grefenstette, Phil Blunsom
ICML 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on the MXM Song Lyrics, 20News Groups and Reuters News datasets demonstrate the effectiveness and efficiency of these neural topic models. |
| Researcher Affiliation | Collaboration | Yishu Miao 1 Edward Grefenstette 2 Phil Blunsom 1 2 1University of Oxford, Oxford, United Kingdom 2Deep Mind, London, United Kingdom. |
| Pseudocode | Yes | Algorithm 1 Unbounded Recurrent Neural Topic Model |
| Open Source Code | No | The paper does not contain any explicit statement about releasing the source code for the methodology described, nor does it provide a link to a code repository. |
| Open Datasets | Yes | We perform an experimental evaluation employing three datasets: MXM2 song lyrics, 20News Groups3 and Reuters RCV1-v24 news. 2http://labrosa.ee.columbia.edu/millionsong/musixmatch (Bertin-Mahieux et al., 2011) 3http://qwone.com/ jason/20Newsgroups 4http://trec.nist.gov/data/reuters/reuters.html |
| Dataset Splits | No | The paper explicitly states training and testing dataset sizes for MXM, 20News Groups, and RCV1-v2 (e.g., 'MXM is the official lyrics collection of the Million Song Dataset with 210,519 training and 27,143 testing datapoints respectively'), but does not specify a validation split. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions using 'Adam (Kingma & Ba, 2015)' for training and 'one layer LSTM cell' for the recurrent network, but does not provide specific version numbers for software libraries or dependencies (e.g., 'PyTorch 1.9', 'Python 3.8'). |
| Experiment Setup | Yes | The hidden dimension of the MLP for constructing q(θ|d) is 256 for all the neural topic models... and 0.8 dropout is applied on the output of the MLP... Grid search is carried out on learning rate and batch size... For the recurrent stick breaking construction we use a one layer LSTM cell (256 hidden units)... The models are trained by Adam (Kingma & Ba, 2015)... For the finite topic models we set the maximum number of topics K as 50 and 200. The truncation-free RSB (RSB-TF) dynamically increases the active topics (γ is empirically set as 5e 5). |