Attentive User-Engaged Adversarial Neural Network for Community Question Answering
Authors: Yuexiang Xie, Ying Shen, Yaliang Li, Min Yang, Kai Lei9322-9329
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the proposed method on large-scale real-world datasets Sem Eval-2016 and Sem Eval-2017. Experimental results verify the benefits of incorporating user information, and show that our proposed model significantly outperforms the stateof-the-art methods. |
| Researcher Affiliation | Collaboration | Yuexiang Xie,1,2 Ying Shen,1 Yaliang Li,2 Min Yang,3 Kai Lei ,1,4 1Shenzhen Key Lab for Information Centric Networking & Blockchain Technology (ICNLAB), SECE, Peking University 2Alibaba Group 3Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences 4PCL Research Center of Networks and Communications, Peng Cheng Laboratory, Shenzhen, China |
| Pseudocode | No | The paper describes the methodology in prose and with diagrams (e.g., Figure 1), but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not state that its source code is publicly available or provide a link to it. |
| Open Datasets | Yes | We conduct the experiment on two widely-adopted public community question answering datasets from Sem Eval-2016 Task 3 (Nakov et al. 2016) and Sem Eval-2017 Task 3 (Nakov et al. 2017). |
| Dataset Splits | Yes | We conduct the experiment on two widely-adopted public community question answering datasets from Sem Eval-2016 Task 3 (Nakov et al. 2016) and Sem Eval-2017 Task 3 (Nakov et al. 2017). We use the English data and the statistics of datasets are summarized in Table 2. In our setting, the sentences are tokenized using NLTK4. Pre-trained Glo VE embeddings of 300 dimensions are adopted as word-level embeddings. Out-of-vocabulary words are set by randomly sampling values from the standard normal distribution. The max length of question, answer and user context are set to 40, 30 and 30 respectively. The hidden size of LSTM is set to 128. The hidden size of discriminator is set to 128. We use Adam optimizer for optimization with learning rate 5e-4. The model parameters are regularized by L2 regularization with the strength of 1e-5. The hyperparameter N is tuned in the set {3,4,5}. Table 2: Statistics of Sem Eval-2016 Task 3 and Sem Eval-2017 Task 3: Sem Eval-2016 (train/dev/test) / Sem Eval-2017 (train/dev/test) The Number of Questions 4,880/244/327 / 5,207/244/293 The Number of Answers 36,198/2,440/3,270 / 39,468/2,440/2,930 Average Length of Questions 43/47/48 / 43/47/54 Average Length of Answers 38/36/37 / 38/36/40 |
| Hardware Specification | No | The paper does not specify the hardware (e.g., CPU, GPU models) used for running the experiments. |
| Software Dependencies | No | The paper mentions some software components (NLTK, GloVe, Adam optimizer) and a URL for NLTK (http://www.nltk.org/) but does not provide specific version numbers for them or other key libraries, which is necessary for reproducibility. For example, it says "Pre-trained Glo VE embeddings" but not the version, and "Adam optimizer" without details. |
| Experiment Setup | Yes | In our setting, the sentences are tokenized using NLTK4. Pre-trained Glo VE embeddings of 300 dimensions are adopted as word-level embeddings. Out-of-vocabulary words are set by randomly sampling values from the standard normal distribution. The max length of question, answer and user context are set to 40, 30 and 30 respectively. The hidden size of LSTM is set to 128. The hidden size of discriminator is set to 128. We use Adam optimizer for optimization with learning rate 5e-4. The model parameters are regularized by L2 regularization with the strength of 1e-5. The hyperparameter N is tuned in the set {3,4,5}. |