Graph-enhanced Optimizers for Structure-aware Recommendation Embedding Evolution
Authors: Cong Xu, Jun Wang, Jianyong Wang, Wei Zhang
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments over six public datasets have demonstrated that it can effectively inject structural information to boost recommendation performance. It is important to note that SEvo does not alter the inference logic of the model, so the inference time is exactly the same and very little computational overhead is required during training. In this section, we comprehensively verify the superiority of SEvo. |
| Researcher Affiliation | Academia | East China Normal University Tsinghua University |
| Pseudocode | Yes | Algorithm 1: SEvo-enhanced Adam W. Differences from the original Adam W are colored in blue. The matrix operation below are element-wise. Algorithm 2: Adam enhanced by SEvo. Differences from the original Adam are colored in blue. The matrix operation below are element-wise. Algorithm 3: SGD with momentum enhanced by SEvo. Differences from the original SGD are colored in blue. The matrix operation below are element-wise. Algorithm 4: Python-style algorithm for similarity estimation based on interaction data. |
| Open Source Code | Yes | Our code is available at https://github.com/MTand HJ/SEvo. |
| Open Datasets | Yes | Datasets. Six benchmark datasets are considered in this paper. The first four datasets including Beauty, Toys, Tools, and Movie Lens-1M are commonly employed in previous studies for empirical comparisons. Additionally, two larger-scale datasets, Clothing and Electronics, are used to assess SEvo s scalability in scenarios involving millions of nodes. ... 2https://cseweb.ucsd.edu/~jmcauley/datasets/amazon/links.html 3https://cseweb.ucsd.edu/~jmcauley/datasets/amazon_v2 4https://grouplens.org/datasets/movielens/1m |
| Dataset Splits | Yes | Following [22, 13], we filter out users and items with less than 5 interactions, and the validation set and test set are split in a leave-one-out fashion, namely the last interaction for testing and the penultimate one for validation. |
| Hardware Specification | Yes | The wall time (seconds) here is evaluated on an Intel Xeon E5-2620 v4 platform and a single GTX 1080Ti GPU, while the results in Table 3 are tested on an Intel Xeon CPU E5-2680 v4 platform and a single RTX 3090 GPU. |
| Software Dependencies | No | The paper mentions implementing models (e.g., GRU4Rec, SASRec) and using optimizers (Adam, SGD), but it does not specify exact version numbers for programming languages, libraries, or frameworks (e.g., Python version, PyTorch/TensorFlow version, CUDA version). |
| Experiment Setup | Yes | Since the purpose is to study the effectiveness of SEvo, only hyperparameters concerning optimization are retuned, including learning rate ([1e-4, 5e-3]), weight decay ([0, 0.1]) and dropout rate ([0, 0.7]). Other hyperparameters in terms of the architecture are consistent with the corresponding baseline. For a fair comparison, the number of layers L is fixed to 3 as in other GNN-based recommenders. As for the hyperparameter in terms of the degree of smoothness, we found β = 0.99 performs quite well in practice. The loss functions follow the suggestions in the respective papers, given that SEvo can be applied to any of them. |