word2ket: Space-efficient Word Embeddings inspired by Quantum Entanglement
Authors: Aliakbar Panahi, Seyran Saeedi, Tom Arodz
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical evidence from three NLP tasks shows that the new word2ket embeddings offer high space saving rate at little cost in terms of accuracy of the downstream NLP model. In order to evaluate the ability of the proposed space-efficient word embeddings in capturing semantic information about words, we used them in three different downstream NLP tasks: text summarization, language translation, and question answering. |
| Researcher Affiliation | Academia | Aliakbar Panahi Department of Computer Science Virginia Commonwealth University panahia@vcu.edu Seyran Saeedi Department of Computer Science Virginia Commonwealth University saeedis@vcu.edu Tom Arodz Department of Computer Science Virginia Commonwealth University Richmond, VA 23284, USA tarodz@vcu.edu |
| Pseudocode | No | The paper describes methods mathematically and conceptually but does not include a structured pseudocode block or an algorithm box. |
| Open Source Code | Yes | 1Py Torch implementation available at: https://github.com/panaali/word2ket |
| Open Datasets | Yes | In text summarization experiments, we used the GIGAWORD text summarization dataset (Graff et al., 2003) using the same preprocessing as (Chen et al., 2019), that is, using 200K examples in training. ... The second task we explored is German-English machine translation, using the IWSLT2014 (DEEN) dataset of TED and TEDx talks as preprocessed in (Ranzato et al., 2016). ... The third task we used involves the Stanford Question Answering Dataset (SQu AD) dataset. |
| Dataset Splits | Yes | We used the validation set to select the best model epoch, and reported results on a separate test set. |
| Hardware Specification | Yes | Each run was executed on a single NVIDIA Tesla V100 GPU card, on a 2 Intel Xeon Gold 6146 CPUs, 384 GB RAM machine. |
| Software Dependencies | No | The paper mentions 'Py Torch implementation' but does not specify its version number or any other software dependencies with explicit version details. |
| Experiment Setup | Yes | In both the encoder and the decoder we used internal layers with dimensionality of 256 and dropout rate of 0.2, and trained the models, starting from random weights and embeddings, for 20 epochs. and We trained the model for 40 epochs, starting from random weights and embeddings, and reported the test set F1 score. |