Bivariate Beta-LSTM
Authors: Kyungwoo Song, JoonHo Jang, Seung jae Shin, Il-Chul Moon5818-5825
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the effectiveness of the bivariate Beta gate structure on the sentence classification, image classification, polyphonic music modeling, and image caption generation. Experiments We compare our models and baselines, LSTM, CIFG-LSTM, G2-LSTM, simple recurrent unit (SRU) (Lei et al. 2018), R-transformer (Wang et al. 2019), Batch normalized LSTM (BN-LSTM) (Cooijmans et al. 2017), and h-detach (Kanuparthi et al. 2019). First, we evaluate the performance of the b Beta-LSTM variants to measure the improvements from our structured gate modeling with the text classifications quantitatively and qualitatively on benchmark datasets. Second, we compare the models on polyphonic music modeling to check the performance of multi-label prediction tasks. Third, we evaluate our models on a pixel-by-pixel MNIST dataset to confirm that our model can alleviate the gradient vanishing problems, empirically. Finally, we perform the image caption generation task to check the performance on the multi-modal dataset. |
| Researcher Affiliation | Academia | Kyungwoo Song, Joon Ho Jang, Seung jae Shin, Il-Chul Moon Korea Advanced Institute of Science and Technology (KAIST), Korea {gtshs2, adkto8093, tmdwo0910, icmoon}@kaist.ac.kr |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks, only mathematical equations and descriptive text. |
| Open Source Code | Yes | Our source code is available at https://github.com/gtshs2/Beta LSTM. |
| Open Datasets | Yes | We compare our models on six benchmark datasets, customer reviews (CR), sentence subjectivity (SUBJ), movie reviews (MR), question type (TREC), opinion polarity (MPQA), and Stanford Sentiment Treebank (SST). We use four polyphonic music modeling benchmark datasets: JSB Chorales, Muse, Nottingham, and Piano. The pixel-by-pixel MNIST task Microsoft COCO dataset (MS-COCO) (Lin et al. 2014). |
| Dataset Splits | Yes | For the experiment, we split the dataset into 80,000, 5,000, 5,000 for the train, the validation, and the test dataset, respectively (Karpathy and Li 2015). |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies or version numbers (e.g., library names with version numbers) required to replicate the experiments. |
| Experiment Setup | Yes | For LSTM models, we use a two-layer structure with 128 hidden dimensions for each layer, following (Lei et al. 2018). For the LSTM baseline, we use a single-layer model with 128 hidden dimensions with Adam optimizer. We use 512 hidden dimensions for the conditional caption generation, and we also used Resnet152 to retrieve image feature vectors. |