Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation
Authors: Aditya Mate, Bryan Wilder, Aparna Taneja, Milind Tambe
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the value of our approach through empirical experiments on synthetic, semisynthetic as well as real case study data and show improved estimation accuracy across the board. |
| Researcher Affiliation | Collaboration | 1School of Engineering and Applied Science, Harvard University, USA 2Google Research, India 3Carnegie Mellon University, USA. |
| Pseudocode | Yes | Algorithm 1 Estimation through Assignment Permutation, Algorithm 2 Reshuffling between index based policies |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code or links to a code repository. |
| Open Datasets | Yes | We use real tuberculosis medication adherence monitoring data, consisting of daily records of patients in Mumbai, India, obtained from (Killian et al., 2019) and by considering an actual large-scale RCT reported in (Mate et al., 2022) evaluating a Restless Multi-Armed Bandit-based algorithm for resource allocation in a maternal and child healthcare. |
| Dataset Splits | No | The paper describes simulating data and running trial instances but does not provide specific train/validation/test dataset splits, percentages, or sample counts for model training and evaluation. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or cloud instance types) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers for reproducibility. |
| Experiment Setup | Yes | We simulate N = 1000 patients in each arm and vary the budget constraint. We consider both the multi-step setting (T = 10) and single step (T = 1). We simulate 300 P1 individuals, 300 P2 individuals and (300 η P3) individuals and set an intervention budget of 300 per timestep for T = 20 timesteps. |