Unsupervised Post-Processing of Word Vectors via Conceptor Negation
Authors: Tianlin Liu, Lyle Ungar, João Sedoc6778-6785
AAAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the post-processed word vectors on a battery of intrinsic lexical evaluation tasks, showing that the proposed method consistently outperforms existing state-of-the-art alternatives. We also show that post-processed word vectors can be used for the downstream natural language processing task of dialogue state tracking, yielding improved results in different dialogue domains. |
| Researcher Affiliation | Academia | Tianlin Liu Department of Computer Science and Electrical Engineering Jacobs University Bremen 28759 Bremen, Germany t.liu@jacobs-university.de Lyle Ungar, Jo ao Sedoc Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104 {ungar, joao}@cis.upenn.edu |
| Pseudocode | Yes | Algorithm 1: The all-but-the-top (ABTT) algorithm for word vector post-processing. Algorithm 2: The conceptor negation (CN) algorithm for word vector post-processing. |
| Open Source Code | Yes | Our codes are available at https://github.com/liutianlin0121/Conceptor-Negation-WV |
| Open Datasets | Yes | We use the publicly available pre-trained Google News Word2Vec (Mikolov et al. 2013)5 and Common Crawl Glo Ve6 (Pennington, Socher, and Manning 2014) to perform lexical-level experiments. |
| Dataset Splits | No | The paper mentions training data for the NBT task ('600 dialogues for training NBT task') and test data ('The test data was Wizard-of-Oz (WOZ) 2.0'), but it does not explicitly specify a validation set or its size/split. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory, or cloud resources) used for running the experiments. |
| Software Dependencies | No | The paper mentions using the Neural Belief Tracker (NBT) model and provides a link to its GitHub repository, but it does not specify version numbers for NBT or any other software dependencies like Python, PyTorch, TensorFlow, or specific libraries. |
| Experiment Setup | Yes | For CN, we fix α = 2 for Word2Vec and Glo Ve throughout the experiments7. For ABTT, we set d = 3 for Word2Vec and d = 2 for Glo Ve, as what has been suggested by Mu and Viswanath (2018). In our experiment with NBT, we used the model specified in (Mrkˇsi c and Vuli c 2018) with default hyper-parameter settings8. To avoid the influences of initial centroids in k-means (...), in this work, we simply fixed the initial centroids as the average of original, ABTT-processed, and CN-processed word vectors respectively from ground-truth categories. |