Music Recommenders: User Evaluation Without Real Users?
Authors: Susan Craw, Ben Horsburgh, Stewart Massie
IJCAI 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that its findings are consistent with those from an online study with real users. This paper explores system-centric evaluation based on user data from social media sites relevant to music. |
| Researcher Affiliation | Academia | Susan Craw and Ben Horsburgh and Stewart Massie IDEAS Research Institute and School of Computing Science & Digital Media Robert Gordon University, Aberdeen, UK {s.craw, s.massie}@rgu.ac.uk |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | The Million Song Dataset [MSD, nd; Bertin-Mahieux et al., 2011] is a step in the right direction, but provides audio data as pre-extracted features, and so cannot be used to evaluate recommenders based on other audio features. Our music collection is Horsburgh s [2013] dataset containing 3174 audio tracks by 764 separate artists. |
| Dataset Splits | No | The paper does not specify explicit training, validation, and test dataset splits with percentages or sample counts for model training. It describes a 'user study' as a 'benchmark to validate' the social Sim evaluation, but this is a conceptual validation, not a data split for model training. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., CPU, GPU models, memory, or cloud instances) used for running the experiments. |
| Software Dependencies | No | The paper mentions software like 'Last.fm s API' and 'Vamp plugin' but does not provide specific version numbers for these or any other software dependencies, which would be necessary for reproducible setup. |
| Experiment Setup | Yes | A k-NN nearest-neighbour retrieval using cosine similarity in the MFS-LSI space identifies the K = 40 most similar audio tracks. The number of pseudo-tags P from p to be included in the hybrid vector h, to balance the number of existing tags T in t. A weighting α determines the influence of selected pseudo-tags p on h1. h = α p + (1 α)t where P = 100 T α = 0.5 P/100 |