Towards Reliable Item Sampling for Recommendation Evaluation
Authors: Dong Li, Ruoming Jin, Zhenming Liu, Bin Ren, Jing Gao, Zhi Liu
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental results confirm our statistical analysis and the superiority of the proposed works. This study helps lay the theoretical foundation for adopting item sampling metrics for recommendation evaluation and provides strong evidence for making item sampling a powerful and reliable tool for recommendation evaluation. ... We perform a thorough experimental evaluation of the proposed item-sampling estimator and the new adaptive sampling method. The experimental results further confirm the statistical analysis and the superiority of newly proposed estimators. |
| Researcher Affiliation | Collaboration | Dong Li1, Ruoming Jin1, Zhenming Liu2, Bin Ren2, Jing Gao3, Zhi Liu3 1 Kent State University, 2 College of William & Mary, 3 i Lambda, {dli12, rjin1}@kent.edu, {zliu, bren}@cs.wm.edu, {jgao, zliu}@ilambda.com |
| Pseudocode | Yes | Algorithm 1: Adaptive Sampling Process |
| Open Source Code | No | The paper does not provide a concrete link or explicit statement about the release of its source code. |
| Open Datasets | Yes | We take three widely-used datasets for recommendation system research, pinterest 20, yelp, ml 20m. See also Appendix for dataset statistics. |
| Dataset Splits | No | The paper does not explicitly provide training/test/validation dataset splits (e.g., percentages or counts) or refer to standard predefined splits for the datasets used. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions common recommendation algorithms (e.g., item KNN, ALS, EASE, Neu MF, Multi VAE) but does not provide specific version numbers for any software dependencies or libraries used for implementation. |
| Experiment Setup | No | The paper mentions experimental setup and uses various models and datasets, but it does not specify concrete hyperparameter values (e.g., learning rate, batch size, epochs, optimizer settings) or other specific training configurations for the experiments. |