STEM: Unsupervised STructural EMbedding for Stance Detection
Authors: Ron Korenblum Pick, Vladyslav Kozhukhov, Dan Vilenchik, Oren Tsur11174-11182
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method on three different datasets from different platforms. Our method outperforms or is comparable with supervised models while providing confidence levels for its output. Furthermore, we demonstrate how the structural embedding relate to the valence expressed by the speakers. |
| Researcher Affiliation | Academia | Ron Korenblum Pick1, Vladyslav Kozhukhov2, Dan Vilenchik2, Oren Tsur1 1Department of Software and Information Science Engineering 2Department of Communication Systems Engineering Ben Gurion University of the Negev {ronpi,kozhukho}@post.bgu.ac.il, {vilenchi, orentsur}@bgu.ac.il |
| Pseudocode | Yes | Algorithm 1: Greedy Speaker Labelling ... Algorithm 2: STEM |
| Open Source Code | Yes | All the source code required for conducting the experiments and reproducing our results is available on Github3 (including the random seed). |
| Open Datasets | Yes | We evaluate our approach on three datasets: Convince Me (Anand et al. 2011), 4Forums (Walker et al. 2012b), and Create Debate (Hasan and Ng 2014). These datasets were used in previous work, e.g., (Walker et al. 2012a; Sridhar et al. 2015; Abbott et al. 2016; Li, Porco, and Goldwasser 2018), among others. |
| Dataset Splits | No | The paper evaluates on datasets and reports results but does not explicitly detail the train/validation/test splits or specific validation set information. |
| Hardware Specification | Yes | We ran the experiment on a machine equipped with a processor with 8 cores and 16GB RAM (we didn t use a GPU for the computation). |
| Software Dependencies | No | To solve the SDP optimization in Eq. (3) we used standard open-source code libraries, PICOS 1 and CVXOPT 2. The paper mentions the libraries but does not specify their version numbers within the text. |
| Experiment Setup | Yes | Our approach uses only two hyperparameters, α (reply weight) and β (quote weight), which are used to compute the weights of the edges in the interaction graph, see Eq. (1). The optimal values may differ between datasets, as the conversational norms may differ. We fixed the values manually; for 4Forum we used α = 0.02, β = 1.0 as participants tend to reply to the OP regardless of the content to which they are replying, and only quote the relevant content instead. For Create Debate and Convince Me we used α = 1.0, β = 0.0 as quotes rarely used. |