Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings
Authors: Na Li, Zied Bouraoui, Jose Camacho-Collados, Luis Espinosa-Anke, Qing Gu, Steven Schockaert
IJCAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We find that a simple strategy of averaging the contextualised embeddings of masked word mentions leads to vectors that outperform the static word vectors learned by BERT, as well as those from standard word embedding models, in property induction tasks. |
| Researcher Affiliation | Academia | 1Nanjing University, China 2CRIL Univ Artois & CNRS, France 3Cardiff University, UK |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Implementation available at https://github.com/lina-luck/rosv ijcai21 |
| Open Datasets | Yes | First, we used the extension of the Mc Rae feature norms dataset [Mc Rae et al., 2005] that was introduced in [Forbes et al., 2019] (XMc Rae5)...5https://github.com/mbforbes/physical-commonsense. Second, we considered CSLB Concept Property Norms6...6https://cslb.psychol.cam.ac.uk/propnorms... Second, we used the Word Net supersenses7...7https://wordnet.princeton.edu/download... As a final dataset, we used the Babel Net domains8 [Camacho Collados and Navigli, 2017]...8http://lcl.uniroma1.et/babeldomains/ |
| Dataset Splits | Yes | For the remaining four datasets, we randomly split the positive examples, for each class, into 60% for training, 20% for tuning and 20% for testing. |
| Hardware Specification | No | The paper does not provide specific hardware details such as exact GPU/CPU models, processor types, or memory amounts used for running the experiments. It only vaguely mentions 'HPC resources'. |
| Software Dependencies | No | The paper mentions software like BERT, RoBERTa, Skip-gram, GloVe, SVM classifier, GCN model, and SVD, but does not provide specific version numbers for any of them. |
| Experiment Setup | Yes | Details about hyper-parameter tuning can be found in the supplementary materials9. For all datasets, we train a binary linear SVM classifier for each of the associated classes. ... The number of negative test examples was chosen as 5 times the number of positive examples. For the training and tuning sets, we randomly select words from the BERT vocabulary as negative examples. The number of negative examples for training was set as twice the number of positive examples. |