Towards Deep Conversational Recommendations
Authors: Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, Chris Pal
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper provides two contributions. First, until now there has been no publicly available large-scale dataset consisting of real-world dialogues centered around recommendations. To address this issue and to facilitate our exploration here, we have collected REDIAL, a dataset consisting of over 10,000 conversations centered around the theme of providing movie recommendations. We make this data available to the community for further research. Second, we use this dataset to explore multiple facets of conversational recommendations. In particular we explore new neural architectures, mechanisms, and methods suitable for composing conversational recommendation systems. Our dataset allows us to systematically probe model sub-components addressing different parts of the overall problem domain ranging from: sentiment analysis and cold-start recommendation generation to detailed aspects of how natural language is used in this setting in the real world. We combine such sub-components into a full-blown dialogue system and examine its behavior. 5 Experiments We propose to evaluate the recommendation and sentiment-analysis modules separately using established metrics. We believe that these individual metrics will improve when modules are more tightly coupled in the recommendation system and thus provide a proxy to overall dialogue quality. We also perform an utterance-level human evaluation to compare responses generated by different models in similar settings. |
| Researcher Affiliation | Collaboration | Raymond Li1, 2, Samira Ebrahimi Kahou1, 3, Hannes Schulz3, Vincent Michalski4, 5, Laurent Charlin5, 6, and Chris Pal1, 2, 5 1Ecole Polytechnique de Montréal 2Element AI 3Microsoft Research Montreal 4Université de Montréal 5Mila 6HEC Montréal |
| Pseudocode | No | The paper describes its models and algorithms using mathematical equations and descriptive text, but does not include any explicit pseudocode blocks or algorithms. |
| Open Source Code | No | The paper states, 'We make this data available to the community for further research,' referring to the REDIAL dataset, but does not provide a statement or link for the open-source code of their models or methodology. |
| Open Datasets | Yes | To address this issue and to facilitate our exploration here, we have collected REDIAL, a dataset consisting of over 10,000 conversations centered around the theme of providing movie recommendations. We make this data available to the community for further research. 1 https://redialdata.github.io/website/ We use the latest Movie Lens dataset6, that contains 26 million ratings across 46,000 movies, given by 270,000 users. 6 https://grouplens.org/datasets/movielens/latest/, retrieved September 2017. |
| Dataset Splits | Yes | Following Sedhain et al. [1], we sampled the training, validation, and test set in a 80-10-10 proportion, and repeated this splitting procedure five times, reporting the average RMSE. Randomly chosen conversations are held out for validation, and each rating, in turn, is predicted using all other ratings (from the same conversation) as inputs. |
| Hardware Specification | No | The paper does not provide any specific details regarding the hardware used for running the experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper mentions various models and optimizers (e.g., GRU, RNN, HRED, Gen Sen, Adam) but does not provide specific version numbers for any software libraries, frameworks, or dependencies used in their implementation. |
| Experiment Setup | No | The paper states, 'In each experiment, for the two training procedures (standard and denoising), we perform a hyper-parameter search on the validation set,' but it does not provide specific hyperparameter values or ranges. It also refers to supplementary materials for more training procedure details, which are not part of the main text. |