Kernel-Based Tests for Likelihood-Free Hypothesis Testing
Authors: Patrik Robert Gerber, Tianze Jiang, Yury Polyanskiy, Rui Sun
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work we (a) introduce a generalization where unlabeled samples come from a mixture of the two classes a case often encountered in practice; (b) study the minimax sample complexity for non-parametric classes of densities under maximum mean discrepancy (MMD) separation; and (c) investigate the empirical performance of kernels parameterized by neural networks on two tasks: detection of the Higgs boson and detection of planted DDPM generated images amidst CIFAR-10 images. For both problems we confirm the existence of the theoretically predicted asymmetric m vs n trade-off. |
| Researcher Affiliation | Academia | Patrik Róbert Gerber Department of Mathematics, MIT Cambridge, MA 02139 prgerber@mit.edu Tianze Jiang Department of Mathematics, MIT Cambridge, MA 02139 tjiang@mit.edu Yury Polyanskiy Department of EECS, MIT Cambridge, MA 02139 yp@mit.edu Rui Sun Department of Mathematics, MIT Cambridge, MA 02139 eruisun@mit.edu |
| Pseudocode | Yes | Algorithm 1 m LFHT with a learned deep kernel |
| Open Source Code | Yes | Our code can be found at https://github.com/Sr-11/LFI. |
| Open Datasets | Yes | detection of the Higgs boson and detection of planted DDPM generated images amidst CIFAR-10 images. |
| Dataset Splits | Yes | Consider splitting the data into three parts: (Xtr, Y tr) is used for training (optimizing) the kernel; (Xev, Y ev) is used to evaluate our test statistic at test time; and (Xcal, Y cal) is used for calibrating the distribution of the test statistic under the null hypothesis. |
| Hardware Specification | Yes | Our code is implemented in Python 3.7 (Py Torch 1.1) and was ran on an NVIDIA RTX 3080 GPU equipped with a standard torch library and dataset extensions. |
| Software Dependencies | Yes | Our code is implemented in Python 3.7 (Py Torch 1.1) and was ran on an NVIDIA RTX 3080 GPU equipped with a standard torch library and dataset extensions. |
| Experiment Setup | Yes | We use 80 training epochs for most of our code from the CNN architecture (for classifiers, this is well after interpolating the training data and roughly when validation loss stops decreasing), and a batch size of 32 which has a slight empirical benefit compared to larger batch sizes. The learning rates are tuned separately in MMD methods for optimality, whereas for classifiers they follow the discriminator s original setting from [40]. In Phase 2 of Algorithm 1, we choose k = 1000 for the desired precision while not compromising runtime. |