Off-policy Evaluation Beyond Overlap: Sharp Partial Identification Under Smoothness
Authors: Samir Khan, Martin Saveski, Johan Ugander
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the effectiveness our methods on two semi-synthetic examples, and obtain informative and valid bounds that are tighter than those possible without smoothness assumptions. |
| Researcher Affiliation | Academia | 1Department of Statistics, Stanford University 2Information School, University of Washington 3Department of Management Science and Engineering, Stanford University. |
| Pseudocode | No | The paper does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code for replicating our results is available at https://github.com/skhan1998/lipschitz-ope. |
| Open Datasets | Yes | We use the Yeast data set from the UCI repository (Dua and Graff, 2017)... Our second experiment uses the Yahoo Webscope s featured news dataset, a standard benchmark for OPE algorithms (Yahoo!, 2011; Li et al., 2010; 2011). |
| Dataset Splits | No | The paper uses the Yeast and Yahoo datasets, but it does not specify explicit train/validation/test splits, percentages, or absolute sample counts for data partitioning to reproduce experiments. It mentions sampling and subsampling but not formal splits. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as CPU/GPU models, processor types, or memory used for running the experiments. |
| Software Dependencies | No | The paper mentions fitting models using 'regularized logistic regression' and refers to 'kernel methods', but it does not specify any software names with version numbers (e.g., Python, PyTorch, scikit-learn, or specific solvers). |
| Experiment Setup | No | The paper describes general experimental settings, such as using a 'regularized logistic regression' for fitting models and varying parameters like 'L' and 'T' for the analysis of bounds. However, it does not provide concrete hyperparameter values (e.g., learning rate, batch size, regularization strength, optimizer settings) or detailed system-level training configurations for these models. |