SensEmBERT: Context-Enhanced Sense Embeddings for Multilingual Word Sense Disambiguation
Authors: Bianca Scarlini, Tommaso Pasini, Roberto Navigli8758-8765
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this Section we report the settings in which we conducted the evaluation of SENSEMBERT when testing it on the English and multilingual WSD tasks. In what follows we introduce the test sets, the system setup along with the reference WSD model, a supervised version of our approach and the comparison systems. |
| Researcher Affiliation | Academia | Bianca Scarlini, Tommaso Pasini, Roberto Navigli Sapienza University of Rome Department of Computer Science {scarlini, pasini, navigli}@di.uniroma1.it |
| Pseudocode | No | The paper describes the approach in steps and includes a figure illustrating the process, but it does not contain formal pseudocode or algorithm blocks. |
| Open Source Code | Yes | The embeddings are released in five different languages at http://sensembert.org. |
| Open Datasets | Yes | As for English, we carried out the evaluation on the test sets in the English WSD framework in Raganato, Camacho-Collados, and Navigli (2017)10. This includes five standardized evaluation benchmarks from the past Senseval-Sem Eval competitions, i.e., Senseval-2 (Edmonds and Cotton 2001), Senseval-3 (Snyder and Palmer 2004), Sem Eval-07 (Pradhan et al. 2007), Sem Eval-13 (Navigli, Jurgens, and Vannella 2013), Sem Eval-15 (Moro and Navigli 2015), together with ALL, the concatenation of the five test sets. |
| Dataset Splits | No | The paper mentions using training and test sets but does not provide specific percentages or counts for validation splits or other dataset split details needed for reproducibility. |
| Hardware Specification | No | The paper mentions using 'BERT large and multilingual models' and specifies their dimensions, but does not provide any details about the specific hardware used to run the experiments (e.g., GPU models, CPU types, memory). |
| Software Dependencies | No | The paper mentions 'BERT large and multilingual models' but does not specify software versions for BERT, PyTorch, TensorFlow, or other libraries used in implementation. |
| Experiment Setup | Yes | We employed two BERT pretrained models: the English 1024-dimensional and the multilingual 768-dimensional pre-trained cased models for the English and multilingual settings, respectively. Among all the configurations reported by Devlin et al. (2019), we used the sum of the last four hidden layers as contextual embeddings of the words. Moreover, BERT exploits Word Piece tokenization, that is, a token can be further split into several subtokens, e.g., the term embed is broken down into two subtokens, namely em and ##bed . Thus, the contextual embedding of an input word was computed as the average of its subtoken embeddings. We used a 1-nearest neighbour approach to test SENSEMBERT on the WSD task. |