Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Combining Machine Learning and Crowdsourcing for Better Understanding Commodity Reviews
Authors: Heting Wu, Hailong Sun, Yili Fang, Kefan Hu, Yongqing Xie, Yangqiu Song, Xudong Liu
AAAI 2015 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we perform real experiments with practical review data to confirm the effectiveness of our method. |
| Researcher Affiliation | Academia | 1School of Computer Science and Engineering, Beihang University, Beijing,China, 100191 2Department of Computer Science, University of Illinois at Urbana-Champaign, United States |
| Pseudocode | No | The paper includes 'Figure 2: The workflow of review analysis' which is a diagram, but it does not contain any structured pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statement or link indicating that the source code for their methodology is open-source or publicly available. |
| Open Datasets | No | The paper states 'We crawl 9,738 reviews of 9 mobile phones from JD.com' indicating a custom collected dataset, but it does not provide any link, citation, or statement about its public availability. |
| Dataset Splits | No | The paper mentions 'The dataset is split into one training set (80%) and one testing set (20%)', but it does not explicitly state the use of a validation set or its split percentage. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments (e.g., GPU/CPU models, memory specifications). |
| Software Dependencies | No | The paper lists the machine learning algorithms used (SVM, NB, KNN, AB, DT) but does not specify any software dependencies or their version numbers (e.g., Python, library versions). |
| Experiment Setup | No | The paper describes the general experimental setup and evaluation metric (F-measure) but does not provide specific details such as hyperparameters (e.g., learning rate, batch size, epochs) or specific optimizer settings for the machine learning algorithms. |