A Unified Model for Financial Event Classification, Detection and Summarization
Authors: Quanzhi Li, Qiong Zhang
IJCAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that our approach outperforms other methods. Experiment result. Table 2, 3 and 4 present the comparison results. R-1, R-2, R-L are the F1 values of ROUGE-1, ROUGE-2 and ROUGE-L, respectively. The result shows that our proposed approach performs better than other methods. To verify if the performance improvement is statistically significant, we conducted t-test between our model and others. The t-test results show that the performance improvements are statistically significant at the level of p=0.05, for the evaluation metrics of event clustering and summarization tasks. It also outperforms the two baseline methods on event type classification, but the difference is not statistically significant. The result also shows that for event clustering UMass is better than LSH, which is similar to the results reported by [Petrovic et al., 2012; Wurzer et al., 2015], in terms of the quality of clusters. The reason is that the focus of LSH is speed, not clustering accuracy. |
| Researcher Affiliation | Industry | Quanzhi Li and Qiong Zhang Alibaba Group {quanzhi.li, qz.zhang}@alibaba-inc.com |
| Pseudocode | No | The paper does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide a link to source code or explicitly state that the code for their methodology is publicly available. |
| Open Datasets | No | Our data set consists of financial news and announcements in English. To ensure the quality of the data set, six annotators were trained on analyzing financial events. The paper describes a custom dataset but provides no specific access information (link, DOI, citation to public repository) for it to be considered publicly available. |
| Dataset Splits | Yes | The data set was split into training, validation and evaluation parts, using a 70/10/20 split. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., CPU, GPU models, memory) used for running the experiments. It only mentions using a pre-trained BERT model and general training settings. |
| Software Dependencies | No | The paper mentions using a 'BERT-base-uncase model' and 'two Adam optimizers' but does not specify version numbers for any key software components, libraries, or frameworks (e.g., TensorFlow, PyTorch versions). |
| Experiment Setup | Yes | We use the BERT-base-uncase model as the encoder for encoding documents. This model uses 12 encoder layers, and the embedding size is 768 for the input token, the position embedding and the text segmentation embedding. The multi-head attention has12 heads, drop out is 0.1, and L2 decay rate is 0.01. The summarization decoder side has six layers, as illustrated in Figure 1. We use two Adam optimizers with β1 = 0.9 and β2 = 0.999 for the encoder and the decoder, respectively, but they have different learning rates and warmup-steps. In Equation 6, we set learn Rate = 2e 3 and warm Up = 30,000 for the BERT encoder, and we set learn Rate = 0.05 and warm Up = 15,000 for the decoder. |