A permutation-free kernel two-sample test
Authors: Shubhanshu Shekhar, Ilmun Kim, Aaditya Ramdas
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We now present experimental validation of the theoretical claims of the previous section. In particular, our experiments demonstrate that (i) the limiting null distribution of x\ MMD 2 is N(0, 1) under a wide range of choices of dimension d, sample sizes n, m and the kernel k, and (ii) the power of our x MMD test is competitive with the kernel-MMD permutation test. We now describe the experiments in more detail. Additional experimental results are reported in Appendix E. |
| Researcher Affiliation | Academia | Shubhanshu Shekhar Department of Statistics and Data Science Carnegie Mellon University Pittsburgh, PA 15213 shubhan2@andrew.cmu.edu Ilmun Kim Department of Statistics and Data Science Department of Applied Statistics Yonsei University ilmun@yonsei.ac.kr Aaditya Ramdas Department of Statistics and Data Science Machine Learning Department Carnegie Mellon University aramdas@stat.cmu.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | We will include the code with the supplemental material. |
| Open Datasets | Yes | The statistic is computed with Gaussian kernel (ksn(x, y) = exp( sn x y 2 2)) with scale parameter sn chosen by the median heuristic for different choices of n, m and d, and with samples X and Y drawn from a multivariate Gaussian distribution with identity covariance matrix. We consider the two-sample testing problem with P = N(0, Id) Q = N(aϵ,j, Id) for different choices of ϵ and d and j. |
| Dataset Splits | No | The paper describes generating synthetic data from specified distributions for experiments rather than using predefined train/validation/test splits of a fixed dataset. |
| Hardware Specification | Yes | The experiments were run on a workstation, whose specifications are provided in Appendix E |
| Software Dependencies | No | The paper does not provide specific software names with version numbers needed to replicate the experiment. |
| Experiment Setup | Yes | The statistic is computed with Gaussian kernel (ksn(x, y) = exp( sn x y 2 2)) with scale parameter sn chosen by the median heuristic for different choices of n, m and d, and with samples X and Y drawn from a multivariate Gaussian distribution with identity covariance matrix. We compare the performance of our proposed test Ψ with the kernel-MMD permutation test, implemented with B = 200 permutations. |