Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Unsupervised Post-Processing of Word Vectors via Conceptor Negation
Authors: Tianlin Liu, Lyle Ungar, João Sedoc6778-6785
AAAI 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the post-processed word vectors on a battery of intrinsic lexical evaluation tasks, showing that the proposed method consistently outperforms existing state-of-the-art alternatives. We also show that post-processed word vectors can be used for the downstream natural language processing task of dialogue state tracking, yielding improved results in different dialogue domains. |
| Researcher Affiliation | Academia | Tianlin Liu Department of Computer Science and Electrical Engineering Jacobs University Bremen 28759 Bremen, Germany EMAIL Lyle Ungar, Jo ao Sedoc Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104 EMAIL |
| Pseudocode | Yes | Algorithm 1: The all-but-the-top (ABTT) algorithm for word vector post-processing. Algorithm 2: The conceptor negation (CN) algorithm for word vector post-processing. |
| Open Source Code | Yes | Our codes are available at https://github.com/liutianlin0121/Conceptor-Negation-WV |
| Open Datasets | Yes | We use the publicly available pre-trained Google News Word2Vec (Mikolov et al. 2013)5 and Common Crawl Glo Ve6 (Pennington, Socher, and Manning 2014) to perform lexical-level experiments. |
| Dataset Splits | No | The paper mentions training data for the NBT task ('600 dialogues for training NBT task') and test data ('The test data was Wizard-of-Oz (WOZ) 2.0'), but it does not explicitly specify a validation set or its size/split. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory, or cloud resources) used for running the experiments. |
| Software Dependencies | No | The paper mentions using the Neural Belief Tracker (NBT) model and provides a link to its GitHub repository, but it does not specify version numbers for NBT or any other software dependencies like Python, PyTorch, TensorFlow, or specific libraries. |
| Experiment Setup | Yes | For CN, we fix α = 2 for Word2Vec and Glo Ve throughout the experiments7. For ABTT, we set d = 3 for Word2Vec and d = 2 for Glo Ve, as what has been suggested by Mu and Viswanath (2018). In our experiment with NBT, we used the model specified in (Mrkˇsi c and Vuli c 2018) with default hyper-parameter settings8. To avoid the influences of initial centroids in k-means (...), in this work, we simply fixed the initial centroids as the average of original, ABTT-processed, and CN-processed word vectors respectively from ground-truth categories. |