Label-Free Explainability for Unsupervised Models
Authors: Jonathan Crabbé, Mihaela van der Schaar
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we conduct quantitative evaluations of the label-free extensions of various explanation methods. We start with simple consistency checks to ensure that these methods provide sensible explanations for unsupervised models. Then, we demonstrate how our label-free explanation paradigm makes it possible to compare representations learned from different pretext tasks. Finally, we challenge Definition 2.1 by studying saliency maps of VAEs. A more detailed description of each experiment can be found in Appendix C. |
| Researcher Affiliation | Academia | 1University of Cambridge 2The Alan Turing Institute 3University of California Los Angeles. |
| Pseudocode | Yes | Algorithm 1 Label-Free Feature Importance |
| Open Source Code | Yes | The implementation is available online4 5. 4https://github.com/Jonathan Crabbe/ Label-Free-XAI 5https://github.com/vanderschaarlab/ Label-Free-XAI |
| Open Datasets | Yes | We fit 3 models on 3 datasets: a denoising autoencoder CNN on the MNIST image dataset (Le Cun et al., 1998), a LSTM reconstruction autoencoder on the ECG5000 time series dataset (Goldberger et al., 2000) and a Sim CLR (Chen et al., 2020) neural network with a Res Net18 (He et al., 2015) backbone on the CIFAR-10 image dataset (Krizhevsky, 2009). |
| Dataset Splits | Yes | The autoencoder is trained for 100 epochs with patience 10 by using Pytorch s Adam with hyperparameters: learning rate = .001, β1 = .9, β2 = .999, ϵ = 10 8, weight decay = 10 5. The testing set is sometimes used for early stopping. This is acceptable because assessing the generalization of the learned model is not the focus of our paper. Rather, we only use the test set to study the explanations of the learned model. ... We train 20 disentangled VAEs of each type for β {1, 5, 10}. (90% 10% traintest split) |
| Hardware Specification | Yes | All our experiments have been performed on a machine with Intel(R) Core(TM) i5-8600K CPU @ 3.60GHz [6 cores] and Nvidia Ge Force RTX 2080 Ti GPU. |
| Software Dependencies | Yes | Our implementation is done with Python 3.8 and Pytorch 1.10.0. |
| Experiment Setup | Yes | All the models are trained to minimize their objective for 100 epochs with patience 10 by using Pytorch s Adam with hyperparameters: learning rate = .001, β1 = .9, β2 = .999, ϵ = 10 8, weight decay = 10 5. |