Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Modeling Selective Feature Attention for Lightweight Text Matching
Authors: Jianxiang Zang, Hui Liu
IJCAI 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental evaluations conducted across diverse text matching baselines and benchmarks underscore the indispensability of modeling feature attention and the superiority of the selection mechanism. Table 1 reports the evaluation accuracies of six lightweight text matching baselines, as well as their performances following the integration of FA and SFA blocks. |
| Researcher Affiliation | Academia | Jianxiang Zang , Hui Liu School of Statistics and Information, Shanghai University of International Business and Economics EMAIL, |
| Pseudocode | No | The paper includes equations and figures illustrating the architecture and processes, but it does not contain a dedicated pseudocode or algorithm block. |
| Open Source Code | Yes | 1Codes available:https://github.com/hggzjx/SFA |
| Open Datasets | Yes | evaluated their performance on following benchmarks: QQP [Iyer et al., 2017], MRPC [Dolan and Brockett, 2005], Bool Q [Clark et al., 2019], SNLI[Bowman et al., 2015], MNLI [Williams et al., 2018](matched&mismatched), QNLI [Wang et al., 2018], and Scitail [Khot et al., 2018]. |
| Dataset Splits | No | The paper evaluates performance on various text matching benchmarks but does not explicitly provide the specific percentages or sample counts for training, validation, and test splits used to reproduce the data partitioning. While Figure 3 refers to 'dev.' for loss curves, it doesn't detail the split ratios. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers (e.g., library or solver names with versions) needed to replicate the experiment. |
| Experiment Setup | No | The paper mentions controlling 'r' and 'N' hyperparameters to manage parameter increment but does not provide specific experimental setup details such as learning rates, batch sizes, number of epochs, or optimizer settings for training. |