MMD-Fuse: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting
Authors: Felix Biggs, Antonin Schrab, Arthur Gretton
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We highlight the applicability of our MMD-FUSE test on both synthetic low-dimensional and real-world high-dimensional data, and compare its performance in terms of power against current state-of-the-art kernel tests. |
| Researcher Affiliation | Academia | Felix Biggs Centre for Artificial Intelligence Department of Computer Science University College London & Inria London contact@felixbiggs.com Antonin Schrab Centre for Artificial Intelligence Gatsby Computational Neuroscience Unit University College London & Inria London a.schrab@ucl.ac.uk Arthur Gretton Gatsby Computational Neuroscience Unit University College London arthur.gretton@gmail.com |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled 'Pseudocode' or 'Algorithm', nor does it present structured steps in a code-like format. |
| Open Source Code | Yes | Our implementation of MMD-FUSE in Jax (Bradbury et al., 2018), as well as the code for the reproducibility of the experiments, are made publicly available at: https://github.com/antoninschrab/mmdfuse-paper |
| Open Datasets | Yes | Perturbed Uniform. In Figure 1, we report test power for detecting perturbations on uniform distributions in one and two dimensions. ... Galaxy MNIST. We examine performance on real-world data in Figure 1, through galaxy images (Walmsley et al., 2022)... CIFAR 10 vs 10.1. The aim of this experiment is to detect the difference between images from the CIFAR-10 (Krizhevsky, 2009) and CIFAR-10.1 (Recht et al., 2019) test sets. |
| Dataset Splits | No | The paper mentions 'parameter selection' for other methods, and that '1000 images from both datasets are used for parameter selection and/or model training' for data-splitting methods, but it does not specify a distinct validation split for its own proposed method, MMD-FUSE, within its experimental setup. |
| Hardware Specification | Yes | The experiments were run on an AMD Ryzen Threadripper 3960X 24 Cores 128Gb RAM CPU at 3.8GHz and on an NVIDIA RTX A5000 24Gb Graphics Card, with a compute time of a couple of hours. |
| Software Dependencies | No | Our implementation of MMD-FUSE in Jax (Bradbury et al., 2018)... While Jax is mentioned, a specific version number for Jax itself is not provided, which is required for reproducibility. |
| Experiment Setup | Yes | For MMD-Fuse, we use Gaussian and Laplace kernels with bandwidths in {qr 5% + i(qr 95% qr 5%)/9 : i = 0, . . . , 9} where qr 5% is half the 5% quantile of all the inter-sample distances { z z r : z, z Z} with r = 1 and r = 2 for Laplace and Gaussian kernels, respectively. Similarly, qr 95% is twice the 95% quantile. ... For MMD-Median, MMD-Split, and MMD-FUSE, we use 2000 permutations to estimate the quantiles. |