Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Fairness in Matching under Uncertainty
Authors: Siddartha Devic, David Kempe, Vatsal Sharan, Aleksandra Korolova
ICML 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate how closely the practical performance aligns with our theoretical results, and how our method compares with prior work, we ran a simple experiment which we detail in Appendix E. Figure 2 summarizes the main take-away: our method enjoys sizable utility gains across different Ο values over a Thompson Sampling baseline. |
| Researcher Affiliation | Academia | 1Department of Computer Science, University of Southern California, Los Angeles, CA 90089. 2Department of Computer Science and School of Public and International Affairs, Princeton University, Princeton, NJ 08544. Correspondence to: Siddartha Devic <EMAIL>. |
| Pseudocode | No | The paper describes algorithmic steps in narrative form and as a numbered list in Section 4.2, but it does not include a formally labeled 'Algorithm' block or 'Pseudocode' figure. |
| Open Source Code | No | The paper does not provide any explicit statements or links indicating that the source code for the methodology is openly available. |
| Open Datasets | Yes | The experiment uses a public dataset from the Czech dating site Libimseti (Brozovsky and Petricek, 2007) which has been used in experiments in prior work on matching (e.g., Su et al. (2022)). |
| Dataset Splits | No | The paper describes the dataset size ('200 users', 'two 100 x 100 matrices') and how parameters were estimated through sampling ('ran the Gale-Shapley algorithm 10,000 times'), but it does not specify any training, validation, or test dataset splits in the conventional machine learning sense. |
| Hardware Specification | No | The paper does not specify any hardware used for running the experiments (e.g., specific GPU or CPU models). |
| Software Dependencies | No | The paper mentions using a 'standard matrix completion technique' and refers to an 'LP solver' but does not provide specific software names with version numbers for reproducibility. |
| Experiment Setup | Yes | To construct the approximate RHS fairness constraints bβin the linear program Equation (2), we ran the Gale-Shapley algorithm 10,000 times, each time with different merits sampled from the aforementioned rating distributions. We set Ο΅ = 0.01, so 10,000 samples is approximately sufficient to obtain error at most Ο΅/2 by Proposition 9. |