Graph-based Dynamic Word Embeddings
Authors: Yuyin Lu, Xin Cheng, Ziran Liang, Yanghui Rao
IJCAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Theoretical analysis and extensive experiments validate the effectiveness of our GDWE on dynamic word embedding learning. We theoretically prove the correctness of using WKGs to assist dynamic word embedding learning and verify the effectiveness of GDWE by cross-time alignment, text stream classification, and qualitative analysis. |
| Researcher Affiliation | Academia | Yuyin Lu1 , Xin Cheng2 , Ziran Liang1 , Yanghui Rao1 1School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China 2School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China |
| Pseudocode | Yes | Algorithm 1 Short-term WKG update algorithm; Algorithm 2 Update WKG(Ga, Gb, δe, δd); Algorithm 3 Long-term WKGs update algorithm. |
| Open Source Code | Yes | Our code and supplementary materials are available in public at: https://github.com/luyy9apples/GDWE. |
| Open Datasets | Yes | We employ two diachronic datasets to compare the performance of different dynamic word embedding models: (1) NYT2: A collection of news from New York Times... (2) Arxiv3: A collection of abstracts from papers published on the Arxiv website... The statistics of all datasets are shown in Table 1. |
| Dataset Splits | Yes | Table 1: Statistics of datasets. Task Dataset #Train #Val #Test #Label Cross-time Alignment NYT 2,205 8,823 - Text Stream Classification NYT 47,423 6,759 12,531 7 NYT (low) 32,475 4,624 9,260 7 Arxiv 1.062,296 151,748 303,506 8 |
| Hardware Specification | Yes | Here, we compare the runtime of our GDWE and baselines by Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz. |
| Software Dependencies | No | The paper mentions software like Word2Vec, BERT, and PyTorch, but does not specify version numbers for any of them (e.g., 'PyTorch 1.9'). |
| Experiment Setup | Yes | For hyper-parameter settings of GDWE and baselines, we uniformly set the window size to 5, the subsampling ratio to 10-4, the number of negative samples to 10. The embedding size of DCWE is 768, which is equal to that of pre-trained BERT embeddings. For GDWE and other baselines, the embedding size is set to 50. We use grid search to determine the best values of other hyper-parameters for GDWE and most of baselines. |