A Kernel Stein Test of Goodness of Fit for Sequential Models
Authors: Jerome Baum, Heishiro Kanagawa, Arthur Gretton
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our test is shown to perform well in practice on discrete sequential data benchmarks. |
| Researcher Affiliation | Academia | 1Department of Computer Science, University College London, UK 2Gatsby Computational Neuroscience Unit, UCL 3School of Mathematics, Statistics and Physics, Newcastle University, UK. |
| Pseudocode | Yes | Algorithm 1 Parametric bootstrap test ... Algorithm 2 Wild bootstrap test |
| Open Source Code | Yes | The code is available at https://github.com/test-for-sequential-models/code |
| Open Datasets | No | The paper describes generating synthetic data for its experiments based on various models (e.g., Markov chains, random walks), but it does not specify public datasets or provide access information (links, DOIs, or citations) for pre-existing public datasets used for training. |
| Dataset Splits | No | The paper performs goodness-of-fit tests on generated samples. It does not mention train/validation/test splits or specific percentages/counts for data partitioning in the context of model training and evaluation. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments, such as GPU/CPU models, memory, or cloud computing instances. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | For the KSD test we use the J-location modification neighborhood (6) with J = ; i.e., we allow edits anywhere in the sequence. ... We choose S = {0, . . . , m 1} for our alphabet, with m = 8. ... The chain terminates with probability 1/20 after each step. ... For the KSD, we use the ZS and ZS configurations as in Section 4.2 with J = 1. We choose sparse, point-dependent insertion and substitution neighborhoods; we use 16 high-probability tokens for a given context. |