Fairness in Ranking under Uncertainty
Authors: Ashudeep Singh, David Kempe, Thorsten Joachims
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In addition to the theoretical characterization, we present an empirical analysis of the potential impact of the approach in simulation studies. For real-world validation, we applied the approach in the context of a paper recommendation system that we built and fielded at the KDD 2020 conference. |
| Researcher Affiliation | Academia | Ashudeep Singh Cornell University Ithaca, NY 14850 ashudeep@cs.cornell.edu David Kempe University of Southern California Los Angeles, CA 90089 david.m.kempe@gmail.com Thorsten Joachims Cornell University Ithaca, NY 14850 tj@cs.cornell.edu |
| Pseudocode | No | The paper describes algorithms and formulations mathematically and textually, but it does not include structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement or link indicating that the source code for their methodology is openly available. |
| Open Datasets | Yes | To evaluate our approach in a recommendation setting with a realistic preference distribution, we designed the following experimental setup based on the Movie Lens 100K (ML-100K) dataset. |
| Dataset Splits | No | The paper mentions taking "10% i.i.d. samples from D to infer the posterior for each movie" for the MovieLens dataset, which relates to data usage, but it does not specify explicit training, validation, and test splits with percentages or sample counts for model evaluation or reproduction. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., CPU, GPU models, memory, or cloud instance types) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies or version numbers (e.g., programming languages, libraries, or solvers with their versions) that would be needed to replicate the experiments. |
| Experiment Setup | Yes | In the experiments presented, we used the ranking policies π , πTS, πMix,φ and πLP,φ to create separate rankings for each of the 18 genres. For each genre, the task is to rank a random subset of 40 movies from that genre. To get a posterior with an interesting degree of uncertainty, we take a 10% i.i.d. samples from D to infer the posterior for each movie. ... The φ-fair policies πLP,φ require the principal to have access to the probabilities Pv Γ h M(v) m,k i which we estimate using 5 104 Monte Carlo samples. |