Towards Equipping Transformer with the Ability of Systematic Compositionality
Authors: Chen Huang, Peixin Qin, Wenqiang Lei, Jiancheng Lv
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results demonstrate that CAT outperforms baselines on compositionality-aware tasks with minimal impact on the effectiveness on standardized language understanding tasks. |
| Researcher Affiliation | Academia | College of Computer Science, Sichuan University, Chengdu, China Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, Chengdu, China |
| Pseudocode | No | The paper describes methods using mathematical equations and diagrams, but no explicit 'Pseudocode' or 'Algorithm' block was found. |
| Open Source Code | No | The paper states 'More details on implementation, datasets and pre-training can be found in the Appendix of our ar Xiv version,' but it does not explicitly state that source code is released or provide a direct link to a code repository in the provided text. |
| Open Datasets | Yes | The Books Corpus (800M words), English Wikipedia (2,500M words), and extra Open How Net dataset serve as our pre-training data for MCAT. We also fine-tune each model on the PAWS (Zhang, Baldridge, and He 2019) and evaluate on Bi RD (Asaadi, Mohammad, and Kiritchenko 2019), IMBD (Maas et al. 2011), SST2 (Socher et al. 2013), STSB-image and STSB-MSRvid (Cer et al. 2017), MNLI-telephone and MNLI-letters (Williams, Nangia, and Bowman 2018), AMAZON-music and AMAZON-video (He and Mc Auley 2016), SQu AD (Rajpurkar et al. 2016), and GLUE (Wang et al. 2018). |
| Dataset Splits | Yes | We selected the best model based on its performance on the validation set for downstream task testing. |
| Hardware Specification | No | The paper mentions 'computational resources' but does not provide specific hardware details such as GPU/CPU models, memory, or cloud instance types used for experiments. |
| Software Dependencies | No | The paper mentions using models like BERT, RoBERTa, and DistilBERT but does not provide specific version numbers for software dependencies such as libraries or frameworks. |
| Experiment Setup | No | The paper states it follows BERT's pre-training process and fine-tunes models with a frozen backbone and MLP layer, but it does not provide specific hyperparameters (e.g., learning rate, batch size, number of epochs, optimizer settings) in the main text. |