Learning to Check Contract Inconsistencies
Authors: Shuo Zhang, Junzhou Zhao, Pinghui Wang, Nuo Xu, Yang Yang, Yiting Liu, Yi Huang, Junlan Feng14446-14453
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments conducted on real-world datasets show the promising performance of our method with a balanced accuracy of 94.05% and an F1 score of 90.90% in the CIC problem. In this section, we conduct experiments on two real-world contract datasets to evaluate our method. We compare the performance of different models, and show the results in Table 2. |
| Researcher Affiliation | Collaboration | 1 MOE KLINNS Lab, Xi an Jiaotong University, Xi an 710049, P. R. China 2 JIUTIAN Team, China Mobile Research |
| Pseudocode | No | The paper describes the model architecture and components in detail with figures, but it does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | 3Codes available at https://github.com/Shuo Zhang XJTU/CIC |
| Open Datasets | Yes | ICAIL Contracts by Chalkidis et al. (2017). These contracts are written in English, and they were used for the task of tagging contract elements. |
| Dataset Splits | No | The paper states it uses '299,621 training samples (blank pairs)' for the Chinese Contracts and '67,765' for ICAIL Contracts, but does not explicitly provide the specific training, validation, and test split percentages or counts needed for reproducibility. It also does not explicitly mention validation sets or their proportions. |
| Hardware Specification | Yes | We implement all the benchmarks using Pytorch on a server equipped with 2 Nvidia Tesla V100 GPUs, each with 32GB memory. |
| Software Dependencies | No | The paper mentions software like 'Pytorch', 'word2vec model', and 'BERT' but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | No | The paper states 'We perform the hyper-parameter search to find the best combinations for all the models. The details are shown in the appendix.' and discusses general hyper-parameter sensitivity (k, N), but does not provide concrete hyperparameter values (e.g., learning rate, batch size, number of epochs, optimizer settings) within the main text. |