Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Deconvolving Feedback Loops in Recommender Systems
Authors: Ayan Sinha, David F. Gleich, Karthik Ramani
NeurIPS 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We use this metric on synthetic and real-world datasets to (1) identify the extent to which the recommender system affects the final rating matrix, (2) rank frequently recommended items, and (3) distinguish whether a user s rated item was recommended or an intrinsic preference. We tested our approach for deconvolving feedback loops on synthetic RS, and designed a metric to identify the ratings most affected by the RS. We then use the same automated technique to study real-world ratings data, and find that the metric is able to identify items influenced by a RS. |
| Researcher Affiliation | Academia | Ayan Sinha Purdue University EMAIL David F. Gleich Purdue University EMAIL Karthik Ramani Purdue University EMAIL |
| Pseudocode | Yes | Algorithm 1: Deconvolving Feedback Loops |
| Open Source Code | No | The paper does not provide any specific links to source code for the methodology or state that code is available. |
| Open Datasets | Yes | Table 1 lists all the datasets we use to validate our approach for deconvolving a RS (from [21, 4, 13]). |
| Dataset Splits | No | The paper describes how synthetic data was generated and the overall evaluation process (e.g., ROC curves), but it does not provide specific details on how the real-world datasets were split into training, validation, and test sets for the experiments. |
| Hardware Specification | No | The paper does not provide any specific details regarding the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., programming languages, libraries, or frameworks) used in the experiments. |
| Experiment Setup | Yes | In our experiment, we draw au N(3, 1), bu N(0.5, 0.5), tu N(0.1, 1), and ηu,i ϵN(0, 1)... We fix the number of iterative updates to be 10, r to be 10 and the resulting rating matrix is Robs. We use α = 1 in all experiments because it models the case when the recommender effects are strong and thus produces the highest discriminative effect between the observed and true ratings (see Figure 2 f). |