Customizing Language Models with Instance-wise LoRA for Sequential Recommendation
Authors: Xiaoyu Kong, Jiancan Wu, An Zhang, Leheng Sheng, Hui Lin, Xiang Wang, Xiangnan He
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on three benchmark datasets demonstrate the effectiveness of i Lo RA, highlighting its superior performance compared to existing methods in mitigating negative transfer and improving recommendation accuracy. Our data and code are available at https://github.com/AkaliKong/iLoRA. |
| Researcher Affiliation | Collaboration | Xiaoyu Kong1 Jiancan Wu1 An Zhang2 Leheng Sheng2 Hui Lin3 Xiang Wang1 Xiangnan He1 1Mo E Key Lab of BIPC, University of Science and Technology of China 2National University of Singapore 3Electronic Science Research Institute of China Electronics Technology Group Corporation |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our data and code are available at https://github.com/AkaliKong/iLoRA. |
| Open Datasets | Yes | We conduct extensive experiments on three benchmark sequential-recommendation datasets (i.e., Last FM [25], Movie Lens [26], Steam [27]) |
| Dataset Splits | No | The paper describes how candidate sets are constructed for evaluation and provides training protocols and hyperparameters, but it does not explicitly state training, validation, or test dataset splits as percentages or sample counts within the paper. |
| Hardware Specification | Yes | Experiments for traditional sequential recommendation baseline models are conducted on a single Nvidia A40, while our i Lo RA framework is implemented on a single Nvidia A100. |
| Software Dependencies | Yes | All experiments are carried out using Python 3.8 and Pytorch Lightning 1.8.6. |
| Experiment Setup | Yes | For all conventional sequential recommendation baselines, we employ the Adam optimization algorithm, establishing a learning rate of 0.001, an embedding dimension of 64, and a batch size of 256, respectively. Furthermore, we incorporate L2 regularization, and the regularization coefficient is fine-tuned through a grid search over a specified set of possible values. For experiments involving methods based on large language models (LLMs), we incorporate a warm-up strategy for the learning rate, which begins the training process with an initial rate set at a fraction of the maximum learning rate. Specifically, the maximum learning rates are set to 2e 4 for the Last FM dataset and 1e 4 for the Movie Lens and Steam datasets. |