AmazonQA: A Review-Based Question Answering Task
Authors: Mansi Gupta, Nitish Kulkarni, Raghuveer Chanda, Anirudha Rayasam, Zachary C. Lipton
IJCAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate numerous models for answer generation and propose strong baselines, demonstrating the challenging nature of this new task. |
| Researcher Affiliation | Academia | Mansi Gupta , Nitish Kulkarni , Raghuveer Chanda , Anirudha Rayasam and Zachary C Lipton Carnegie Mellon University {mansig1, nitishkk, rchanda, arayasam, zlipton}@cs.cmu.edu |
| Pseudocode | No | The paper describes models and processes but does not include any pseudocode or algorithm blocks. |
| Open Source Code | Yes | To promote research in this direction, we publicly release2 the dataset and our implementations of all baselines. 2https://github.com/amazonqa/amazonqa |
| Open Datasets | Yes | We build upon the dataset of [Mc Auley and Yang, 2016]... We make a random 80-10-10 (train-development-test) split. To promote research in this direction, we publicly release2 the dataset and our implementations of all baselines. 2https://github.com/amazonqa/amazonqa |
| Dataset Splits | Yes | We make a random 80-10-10 (train-development-test) split. Each split uniformly consists of 85% of descriptive and 15% yes/no question types. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments (e.g., CPU, GPU models, or memory specifications). |
| Software Dependencies | No | The paper mentions using specific models like LSTM-based encoders/decoders and R-Net, and evaluation metrics like BLEU and ROUGE, but it does not specify version numbers for any software dependencies or libraries used. |
| Experiment Setup | No | The paper describes model architectures (e.g., LSTM-based encoder-decoder), feature engineering for the answerability classifier, and various baseline models. However, it does not provide specific hyperparameters like learning rates, batch sizes, number of epochs, or optimizer settings, which are crucial for reproducing the experiments. |