On the Adversarial Robustness of Benjamini Hochberg
Authors: Louis Chen, Roberto Szechtman, Matan Seri
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | we perform computational experiments. (Abstract), In Section 5 we provide accompanying numerical experiments on i.i.d. as well as PRDS p-values. (Section 1.5), In this section, we provide computations (performed in R and Python...) to demonstrate the performance of the adversarial algorithm INCREASE-c. (Section 5) |
| Researcher Affiliation | Academia | Louis L Chen Operations Research Department Naval Postgraduate School Monterey, CA 93943 louis.chen@nps.edu Roberto Szechtman Operations Research Department Naval Postgraduate School Monterey, CA 93943 rszechtm@nps.edu Matan Seri Operations Research Department Naval Postgraduate School Monterey, CA 93943 matan.seri@gmail.com |
| Pseudocode | Yes | INCREASE-c runs as follows: 1. IF B0 N+1 c, then move the largest c (ties broken arbitrarily) in the (N+1)-th bin to bin k+c. 2. ELSE leave the p-values unperturbed. (Section 3) |
| Open Source Code | No | The paper does not provide an explicit statement or link to the open-source code for the methodology described in this paper. |
| Open Datasets | Yes | The Credit Card2 dataset D = {(Xi,Yi)}n i=1 R30 {0,1} contains n = 284,807 credit card transactions in September 2013 by European cardholders over the course of two days 492 of which were frauds. Each Xi R30 consists of numerical input variables that are the result of a PCA transformation. The Class" label Yi takes value 1 in case of fraud and 0 otherwise (/genuine), yielding the partition [n] = n0 n1, with n0 = 284,315 and n1 = 492. website: https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud (Section 5.2) |
| Dataset Splits | Yes | From the set n0 of genuine transactions, we uniformly at random select a subset ntrain n0 of size 141,758 to form a training set Dtrain = {Xi}i ntrain... Then for each of 102 simulations, we uniformly at random selected a subset ncal n0 ntrain of size 141,657 to form a calibration set Dcal = {Xi}i ncal... to form a test set Dtest = {Xi}i ntest (Section 5.2) |
| Hardware Specification | Yes | computations (performed in R and Python on a Macbook Air-M2 chip, 8GB memory, with no experiment time exceeding 5 minutes) (Section 5) |
| Software Dependencies | No | The paper mentions 'R and Python' and 'the R library isotree3' but does not specify version numbers for these software components or any other libraries. |
| Experiment Setup | Yes | Following the framework from Remark 4.6, we simulated 104 replications of the following experiment: (1) N = 1000 p-values are generated, with {pi}i N0 iid U(0,1), and each pi among {pi}i N1 gen-erated via pi = 1 Φ( zi µ1 1 ), with {zi}i N0 iid N(µ1,1); (2) FDP[BHq;p] and FDP[BHq;p+c] are calculated. (Section 5.1.1) and N = 103, q = 0.10, π0 = 0.90, and all σi = 1. (Table 1 caption). |