Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
The Weighted Generalised Covariance Measure
Authors: Cyrill Scheidegger, Julia Hörrmann, Peter Bühlmann
JMLR 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We introduce a new test for conditional independence which is based on what we call the weighted generalised covariance measure (WGCM). It is an extension of the recently introduced generalised covariance measure (GCM). To test the null hypothesis of X and Y being conditionally independent given Z, our test statistic is a weighted form of the sample covariance between the residuals of nonlinearly regressing X and Y on Z. We propose different variants of the test for both univariate and multivariate X and Y . We give conditions under which the tests yield the correct type I error rate. Finally, we compare our novel tests to the original GCM using simulation and on real data sets. Typically, our tests have power against a wider class of alternatives compared to the GCM. |
| Researcher Affiliation | Academia | Cyrill Scheidegger EMAIL Seminar for Statistics ETH Z urich 8092 Z urich, Switzerland Julia H orrmann EMAIL Department of Computer Science Seminar for Statistics ETH Z urich 8092 Z urich, Switzerland Peter B uhlmann EMAIL Seminar for Statistics ETH Z urich 8092 Z urich, Switzerland |
| Pseudocode | Yes | Method 1 (WGCM.est) Using one random sample split, create two independent data sets (X(n), Y(n), Z(n)) and A = (XA, YA, ZA). We use the data set A to estimate a weight function and calculate the test statistic on the data set (X(n), Y(n), Z(n)) as in Section 2.1. For ease of notation, we still assume that (X(n), Y(n), Z(n)) consists of n samples, whereas the size an of the data set A = (XA, YA, ZA) is arbitrary (but depends on n). The ratio between the sizes of the data sets (X(n), Y(n), Z(n)) and A is difficult to choose in practice. We propose to estimate a weight function as follows: 1. (Nonlinearly) regress XA on ZA to get ˆf A and YA on ZA to get ˆg A. Let ˆϵA,i = x A,i ˆf A(z A,i) and ˆξA,i = y A,i ˆg A(z A,i). 2. (Nonlinearly) regress ˆϵA,iˆξA,i an i=1 on ZA to get ˆh which is an estimate of h( ) = EP [ϵξ|Z = ]. 3. Set ˆw(n)( ) = sign(ˆh( )). |
| Open Source Code | Yes | Our implementations are based on the functions from the package Generalised Covariance Measure, see Peters and Shah (2019). Our code is available as the R-package weighted GCM on CRAN. |
| Open Datasets | Yes | As a first example, we analyse the Boston housing data, see Harrison and Rubinfeld (1978). Among the set of 13 predictors, we want to find the most relevant ones to predict the target variable medv, which is the median value of owner-occupied homes. ... We analyse the online news popularity data set, see Fernandes et al. (2015). The data can be obtained from the UCI Machine Learning Repository, see Dua and Graff(2017). ... We look at the wave energy converters data set, available at the UCI Machine Learning Repository (Dua and Graff, 2017). |
| Dataset Splits | Yes | Method 1 (WGCM.est) Using one random sample split, create two independent data sets (X(n), Y(n), Z(n)) and A = (XA, YA, ZA). ... (WGCM.est) The WGCM with one single estimated weight function, where 30% of the samples are used to estimate the weight function. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or processor types) used for running the experiments. |
| Software Dependencies | No | Our implementations are based on the functions from the package Generalised Covariance Measure, see Peters and Shah (2019). Our code is available as the R-package weighted GCM on CRAN. ... We perform all three tests both with regression splines using gam from the R package mgcv (see Wood, 2017) and with boosted regression trees using the package xgboost (see Chen and Guestrin, 2016 and Chen et al., 2021). While the xgboost reference lists a version (R package version 1.4.1.1), the main text does not specify versions for 'mgcv' or 'Generalised Covariance Measure' and does not explicitly list multiple key software components with versions in the experimental setup description. |
| Experiment Setup | Yes | In Figure 1, we plot the rejection rates at level α = 0.05 for the three methods GCM, WGCM.est and WGCM.fix for different values of λ [0, 1]. The rejection rates are calculated from 1000 simulation runs with n = 200 samples. ... (WGCM.est) The WGCM with one single estimated weight function, where 30% of the samples are used to estimate the weight function. ... (WGCM.fix) ... We will use k0 = 7. This means we have a total of 8 weight functions in the setting (1D) and 71 weight functions in the settings (10D.Add) and (10D.Non Add). ... For every combination of b1, b2 {0, 1/3, 1}, we simulate 500 data sets with n samples for each setting and perform the following tests: (GCM) The (unweighted) GCM by Shah and Peters (2020). (WGCM.est) The WGCM with one single estimated weight function... (WGCM.fix) The WGCM with fixed weight functions... We perform all three tests both with regression splines using gam from the R package mgcv (see Wood, 2017) and with boosted regression trees using the package xgboost (see Chen and Guestrin, 2016 and Chen et al., 2021). |