Making the Cut: A Bandit-based Approach to Tiered Interviewing
Authors: Candice Schumann, Zhi Lang, Jeffrey Foster, John Dickerson
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show via simulations on real data from one of the largest US-based computer science graduate programs that our algorithms make better hiring decisions or use less budget than the status quo. |
| Researcher Affiliation | Academia | University of Maryland Tufts University {schumann,zlang}@cs.umd.edu, jfoster@cs.tufts.edu, john@cs.umd.edu |
| Pseudocode | Yes | Algorithm 1 provides pseudocode for CACO. Algorithm 2 provides pseudocode for BRUTAS. |
| Open Source Code | No | The paper does not provide concrete access to source code (specific repository link, explicit code release statement, or code in supplementary materials) for the methodology described. |
| Open Datasets | No | The dataset used is described as "real admissions data from one of the largest US-based graduate computer science programs" and "Our dataset consists of three years (2014 16) worth of graduate applications." but no information regarding its public availability or access is provided. |
| Dataset Splits | Yes | Using information from 2014 and 2015, we used a random forest classifier [Pedregosa et al., 2011], trained in the standard way on features extracted from the applications, to predict probability of acceptance. In the testing phase, the classifier was run on the set of applicants A from 2016... |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions using a "random forest classifier [Pedregosa et al., 2011]" and "Latent Dirichlet Allocation (LDA)", but does not specify any version numbers for these software components or libraries. |
| Experiment Setup | Yes | We determined that the time taken for a Skype interview is roughly 6 times as long as a packet review, and therefore we set the cost multiplier for the second stage j2 = 6. We ran over a variety of s2 values, and we determined σ by looking at the distribution of review scores from past years. When an arm a 2 A is pulled with information gain s and cost j, a reward is randomly pulled from the arm s review scores (when s1 = 1 and j1 = 1, as in the first stage), or a reward is pulled from a Gaussian distribution with mean P(a) and a standard deviation of σ ps. |