Causally motivated multi-shortcut identification and removal
Authors: Jiayun Zheng, Maggie Makar
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show both theoretically and empirically that our approach is able to identify a sufficient set of shortcuts leading to more efficient predictors in finite samples. (4) We empirically validate our theoretical findings using a semi-simulated benchmark and a medical task, showing our approach has favorable inand out-of-distribution generalization properties. |
| Researcher Affiliation | Academia | Jiayun Zheng Computer Science and Engineering University of Michigan, Ann Arbor Maggie Makar Computer Science and Engineering University of Michigan, Ann Arbor |
| Pseudocode | Yes | A full description of the shortcut identification procedure is included in the appendix, section C, procedure 1. Pseudocode for our full approach is included in the appendix section C. |
| Open Source Code | Yes | Our code is available on https://github.com/mymakar/cm_multishortcut_id_removal. (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] The github link that contains code and instructions is listed in section 6. |
| Open Datasets | Yes | We follow Sagawa et al. [36] by constructing a semi-synthetic waterbirds dataset where the task is to predict Y... We combine images of water and land birds extracted from the Caltech-UCSD Birds-200-2011 (CUB) dataset [42] with water and land background extracted from the Places dataset [44]. We use a publicly available dataset made available by Eye PACS, LLC [11]4. (a) If your work uses existing assets, did you cite the creators? [Yes] The creators of all three datasets used in the paper (CUB, Places and DR) are cited in the experiments section. (b) Did you mention the license of the assets? [Yes] We mention in section 6 that all datasets are publicly available. |
| Dataset Splits | Yes | We split the training data into two sub samples D1 and D2. Cross-validation. The objective function in (4) depends on two hyperparameters... Letting Dvalid denote a held out validation set, φvalid denote {φ(xi)}i2Dvalid, and similarly define b Vpvalid, our cross validation procedure proceeds as follows. |
| Hardware Specification | Yes | (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] In the appendix. |
| Software Dependencies | No | All models in this paper are implemented in Tensor Flow [1]. While TensorFlow is mentioned, no specific version number for it or other software dependencies is provided in the main text to ensure reproducibility. |
| Experiment Setup | Yes | Additional details about training are included in the appendix. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] In section 6 and the appendix |