Human Expertise in Algorithmic Prediction
Authors: Rohan Alur, Manish Raghavan, Devavrat Shah
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In an X-ray classification task, we find that this subset constitutes nearly 30% of the patient population. Our approach provides a natural way of uncovering this heterogeneity and thus enabling effective human-AI collaboration. |
| Researcher Affiliation | Academia | Rohan Alur EECS, LIDS MIT ralur@mit.edu Manish Raghavan EECS, LIDS, Sloan MIT mragh@mit.edu Devavrat Shah EECS, IDSS, LIDS, SDSC MIT devavrat@mit.edu |
| Pseudocode | Yes | Algorithm 1 A method for incorporating human expertise into algorithmic predictions |
| Open Source Code | Yes | Code to replicate our experiments is available at https://github.com/ralur/heap-repl. |
| Open Datasets | Yes | These models were trained on a dataset of 224,316 chest radiographs collected across 65,240 patients [35], and then evaluated on a holdout set of 500 randomly sampled radiographs. |
| Dataset Splits | No | The paper mentions a training dataset and a holdout set for evaluation, but does not explicitly detail training, validation, and test splits (e.g., percentages or exact counts for each split, or references to predefined splits) in the main text. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU, GPU models, memory) used for running the experiments in its main text. |
| Software Dependencies | No | The paper mentions using Python and various algorithms but does not provide specific version numbers for any software dependencies like libraries or frameworks. |
| Experiment Setup | No | The paper describes high-level experimental approaches, such as partitioning patients and using specific models, but does not provide concrete hyperparameters (e.g., learning rate, batch size) or detailed system-level training settings in the main text. |