Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Predicting Twitter User Demographics using Distant Supervision from Website Traffic Data
Authors: Aron Culotta, Nirmal Kumar Ravi, Jennifer Cutler
JAIR 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We then fit a regression model to predict these demographics from information about the followers of each website on Twitter. Using patterns derived both from textual content and the social network of each user, our final model produces an average held-out correlation of .77 across seven different variables (age, gender, education, ethnicity, income, parental status, and political preference). |
| Researcher Affiliation | Academia | Aron Culotta EMAIL Nirmal Kumar Ravi EMAIL Department of Computer Science, Illinois Institute of Technology Chicago, IL 60616 Jennifer Cutler EMAIL Stuart School of Business, Illinois Institute of Technology Chicago, IL 60616 |
| Pseudocode | No | The paper describes the methodology using mathematical equations and textual descriptions of steps, such as the formulation of elastic net regularization and multi-task elastic net, but does not present any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | 1. Replication code and data are available here: https://github.com/tapilab/jair-2016-demographics. |
| Open Datasets | Yes | Quantcast.com is an audience measurement company that tracks the demographics of visitors to millions of websites... As of this writing, the estimated demographics of a large number of websites is publicly accessible through their searchable web interface. The gender and ethnicity data were originally collected by Mohammady and Culotta (2014)... The political preference data comes from Volkova, Coppersmith, and Van Durme (2014), who in turn builds on the labeled data of Pennacchiotti and Popescu (2011) and Al Zamal et al. (2012). |
| Dataset Splits | Yes | We perform five-fold cross-validation and report the held-out correlation coefficient (r) between the predicted and true demographic variables. We tuned the regularization parameters on a held-out set of 200 accounts for Gender prediction... We perform three-fold cross-validation to compare accuracy with the distantly supervised approach. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | We use the implementation of Multi Task Elastic Net in scikit-learn (Pedregosa et al., 2011). (Missing version number for scikit-learn and other dependencies.) |
| Experiment Setup | Yes | We tuned the regularization parameters on a held-out set of 200 accounts for Gender prediction, setting the scikit-learn parameters l1 ratio=0.5 for each model, alpha=1e 5 for the Friends model, and alpha=1e 2 for the Text and Friends+Text models. |