Learnable Embedding sizes for Recommender Systems
Authors: Siyi Liu, Chen Gao, Yihong Chen, Depeng Jin, Yong Li
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate it can efficiently cut down embedding parameters and boost the base model s performance. Specifically, it achieves strong recommendation performance while reducing 97-99% parameters. |
| Researcher Affiliation | Academia | 1University of Electronic Science and Technology of China, Chengdu, China 2Beijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University, Beijing 100084, China 3University College London, London, United Kingdom |
| Pseudocode | Yes | Figure 7: Py Torch code of PEP. ... Algorithm 1 Our PEP |
| Open Source Code | Yes | 1Codes are available at: https://github.com/ssui-liu/learnable-embed-sizes-for-Rec Sys |
| Open Datasets | Yes | We use three benchmark datasets: Movie Lens-1M, Criteo, and Avazu, in our experiments. ... 9https://grouplens.org/datasets/movielens 10https://www.kaggle.com/c/criteo-display-ad-challenge 11https://www.kaggle.com/c/avazu-ctr-prediction |
| Dataset Splits | Yes | For each dataset, all the samples are randomly divided into training, validation, and testing set based on the proportion of 80%, 10%, and 10%. |
| Hardware Specification | Yes | We use Py Torch (Paszke et al., 2019) to implement our method and train it with mini-batch size 1024 on a single 12G-Memory NVIDIA TITAN V GPU. |
| Software Dependencies | No | The paper mentions "Py Torch (Paszke et al., 2019)" and "TensorFlow (Abadi et al., 2016)" but does not provide specific version numbers for these software components (e.g., "PyTorch 1.9"). |
| Experiment Setup | Yes | Following Auto Int (Song et al., 2019) and Deep FM (Guo et al., 2017), we employ Adam optimizer with the learning rate of 0.001 to optimize model parameters in both the pruning and re-training stage. ... We use Py Torch (Paszke et al., 2019) to implement our method and train it with mini-batch size 1024... For g(s), we apply g(s) = 1 1+e s in all experiments and initialize the s to 15, 150 and 150 in Movie Lens-1M, Criteo and Avazu datasets respectively. Moreover, the granularity of PEP is set as Dimension-wise for PEP-2, PEP-3, and PEP-4 on Criteo and Avazu datasets. And others are set as Feature Dimension-wise. The base embedding dimension d is set to 64 for all the models before pruning. |