Understanding Composition of Word Embeddings via Tensor Decomposition
Authors: Abraham Frandsen, Rong Ge
ICLR 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also complement our theoretical results with experiments that verify our assumptions, and demonstrate the effectiveness of the new composition method. and Finally, we train our new model on a large corpus and give experimental evaluations. |
| Researcher Affiliation | Academia | Abraham Frandsen & Rong Ge Department of Computer Science Duke University Durham, NC 27708, USA {abef,rongge}@cs.duke.edu |
| Pseudocode | No | The paper describes the models and methods in prose and mathematical equations but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | 2code for preprocessing, training, and experiments can be found at https://github.com/ abefrandsen/syntactic-rand-walk |
| Open Datasets | Yes | We train our model using a February 2018 dump of the English Wikipedia. and We also test our tensor composition method on a adjective-noun phrase similarity task using the dataset introduced by Mitchell & Lapata (2010). and We use the movie review dataset of Pang and Lee (Pang & Lee, 2004) as well as the Large Movie Review dataset (Maas et al., 2011). |
| Dataset Splits | Yes | Following Mitchell & Lapata (2010), we split the data into a development set of 18 humans and a test set of the remaining 36 humans. We use the development set to select the optimal scalar weight for the weighted tensor composition, and using this fixed parameter, we report the results using the test set. and We evaluate the test accuracy of each method using 5-fold cross-validation on the smaller dataset and the training-test set split provided in the larger dataset. |
| Hardware Specification | No | The paper does not specify any hardware details like CPU, GPU models, or memory used for running the experiments. It only mentions using the Tensorflow framework. |
| Software Dependencies | No | The paper mentions Tensorflow framework (Abadi et al., 2016), Adam optimizer (Kingma & Ba, 2014), and scikit-learn (Pedregosa et al., 2011) but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | To reduce the number of parameters, we constrain T to have CP rank 1000. and In all cases, we utilize the Tensorflow framework (Abadi et al., 2016) with the Adam optimizer (Kingma & Ba, 2014) (using default parameters), and train for 1-5 epochs. |