Sliced Mutual Information: A Scalable Measure of Statistical Dependence
Authors: Ziv Goldfeld, Kristjan Greenewald
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our theory is supported by numerical studies of independence testing and feature extraction, which demonstrate the potential gains SMI offers over classic MI for high-dimensional inference. ... We validate our theory on synthetic experiments, demonstrating that SMI is a scalable alternative to classic MI when dealing with high-dimensional data. ... 4 Empirical Results |
| Researcher Affiliation | Collaboration | Ziv Goldfeld Cornell University goldfeld@cornell.edu Kristjan Greenewald MIT-IBM Watson AI Lab kristjan.h.greenewald@ibm.com |
| Pseudocode | Yes | Pseudocode and computational complexity for b SIn,m can be found in Section B of the supplement. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing code for the methodology or a link to a code repository. |
| Open Datasets | Yes | Figure 4: Solution for optimizing A-transformed SMI of the 0-1 MNIST setup using S-MINE. Rows 0 and 1 of A are shown in (a) and (b), respectively. We next combine S-MINE with independence testing, looking to maximize the SMI using transformations AX, AY , where X, Y are samples from a random MNIST class (either 0 or 1) and A 2 R10 d. |
| Dataset Splits | No | The paper describes generating '50 datasets comprising n positive samples' and '50 more dataset of negative samples' for independence testing, and uses Monte-Carlo sampling. However, it does not specify explicit train/validation/test splits with percentages or counts for model training or evaluation in a reproducible manner. |
| Hardware Specification | No | The paper does not explicitly describe the hardware (e.g., specific GPU/CPU models, cloud resources with specifications) used to run its experiments. |
| Software Dependencies | No | The paper mentions using the 'Kozachenko Leonenko estimator', 'MINE', and 'S-MINE', but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | The MC step for SMI estimation (see (6)) uses 1000 random slices... with g in S-MINE realized as a two-layer fully connected neural network with 100 hidden units for. |