Sentiment Analysis via Deep Hybrid Textual-Crowd Learning Model
Authors: Kamran Ghasedi Dizaji, Heng Huang
AAAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results indicate that our model achieves superior results in comparison with the state-of-the-art models, especially when the crowd labels are scarce. In this section, we first evaluate the performance of our hybrid model in the crowd aggregation task, and then examine the quality of the learned language models. In order to compare the proposed model with the state-of-the-art aggregation models, we use two large-scale crowdsourcing datasets, which have text data along with crowd labels for sentiment analysis. |
| Researcher Affiliation | Academia | Kamran Ghasedi Dizaji, Heng Huang Electrical and Computer Engineering, University of Pittsburgh, USA kag221@pitt.edu, heng.huang@pitt.edu |
| Pseudocode | Yes | Algorithm 1: Crowd Deep AE Algorithm Initialize qi by majority voting i {1, ..., N} while not converged do Solve problem (6) to update θ Solve problem (8) to update α Solve problem (9) to update W qic jk (eic)α0(pijck)αj1ijk samples by the denoising autoencoder converges to original data-generating distribution. |
| Open Source Code | No | The paper does not include an unambiguous statement that the authors are releasing the code for the work described, nor does it provide a direct link to a source-code repository. |
| Open Datasets | No | For Crowd Flower (CF) dataset: 'Crowd Flower (CF) dataset was a part of the 2013 Crowdsourcing at Scale shared task challenge, collected by Crowd Flower2 as a rich source for the sentiment analysis of tweets about the weather. The dataset includes 569,375 crowd labels for 98,980 tweets. But the gold-standard (true) labels are only provided for 300 tweets, which correspond to 1720 crowd labels collected from 461 workers. 2www.crowdflower.com'. For Sentiment Polarity (SP) dataset: 'Sentiment Polarity (SP) dataset includes the sentiment analysis of crowd workers about the movie reviews across two categories, fresh (positive) and rotten (negative). The dataset consists of 5,000 sentences from the movie reviews in Rotten Tomatoes website3, which is extracted by (Pang and Lee 2004). 3www.rottentomatoes.com'. While the datasets are named, the paper does not provide direct links or specific access information for the exact datasets used for training beyond general website links for the source platforms or a citation to a paper that describes the dataset, without confirming the dataset's public accessibility or providing a direct download link/repository for the version used. |
| Dataset Splits | No | The paper describes experiments run with varying percentages of available crowd labels (e.g., 2%, 20%, 100%) but does not specify clear train/validation/test dataset splits (e.g., 80/10/10 split or specific sample counts) needed for reproduction. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions software components like Adam optimizer, Xavier initialization, and Porter stemming, but does not provide specific version numbers for these or other key software libraries (e.g., Python, TensorFlow, PyTorch) that would be needed for replication. |
| Experiment Setup | Yes | For the deep autoencoder, we consider three fully connected layers for both encoder and decoder pathways with 512, 256, and 128 neurons as the feature maps, and then add a softmax layer on top of the encoder pathway. The leaky rectified activation (leaky RELU) is used as the activation function for the autoencoder layers, except the reconstruction layer at the end of decoder pathway, which has rectified activation (RELU) to reconstruct text samples. Moreover, we set the learning rate to 10 4 and adopt Adam (Kinga and Adam 2015) as our optimization method. The weights of all layers are also initialized by the Xavier or Glorot Uniform initialization approach (Glorot and Bengio 2010). ... We use a trick in (Tian and Zhu 2015), that employs the nonrelated likelihood for selecting the hyper-parameters. In particular, we utilize the likelihood function p(XCr|Y, θ) to choose λα, λd and dropout from λset α = {0.01, 0.1, 1}, λset d = {0.01, 0.1, 1} and dropoutset = {0.1, 0.2, 0.3}, and adopt p(Y|XT e, W) as a criterion to choose λθ from λset θ = {0.01, 0.1, 1}. |