Do Users Benefit From Interpretable Vision? A User Study, Baseline, And Dataset
Authors: Leon Sixt, Martin Schuessler, Oana-Iuliana Popescu, Philipp Weiß, Tim Landgraf
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conducted a user study (N=240) to test how such a baseline explanation technique performs against concept-based and counterfactual explanations. ... Our results show that the baseline outperformed concept-based explanations. |
| Researcher Affiliation | Academia | Leon Sixt 1, Martin Schuessler 23, Oana-Iuliana Popescu1, Philipp Weiß3, Tim Landgraf1 Freie Universität Berlin1 Weizenbaum Institut Berlin2 TU Berlin3 |
| Pseudocode | Yes | Listing 1: Source code example to create a biased sampler. |
| Open Source Code | Yes | We open-source our dataset, explanation techniques, model, study design, including instructions and videos to support replicating our results as well as adapting our design to other explanation techniques. ... Model export: https://f002.backblazeb2.com/file/iclr2022/do_users_benefit_from_interpretable_vision_model.tar.gz |
| Open Datasets | Yes | We open-source our dataset, explanation techniques, model, study design, including instructions and videos to support replicating our results as well as adapting our design to other explanation techniques. ... Biased dataset: https://f002.backblazeb2.com/file/iclr2022/two4two_obj_color_and_spherical_finer_search_spherical_uniform_0.33_uniform_0.15.tar |
| Dataset Splits | No | B: NN predictions explained with 10 sorted rows of 5 images drawn from the validation set (50 original images). |
| Hardware Specification | No | We thank the Center for Information Services and High Performance Computing (ZIH) at Dresden University of Technology and the HPC Service of ZEDAT, Freie Universität Berlin, for generous allocations of computation time (Bennett et al., 2020). |
| Software Dependencies | No | The paper mentions architectures like Glow and MobileNet V2 but does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | Our model is based on the Glow architecture (Kingma & Dhariwal, 2018) and contains 7 blocks. A block is a collection of 32 flow steps, followed by a down-sampling layer, and ends with a fade-out layer. A single flow step consists of actnorm, invertible 1x1 convolution and affine coupling layer. ... The model is trained using a supervised loss and an unsupervised objective. ... For the supervised loss Lsup, we use the binary cross entropy. As unsupervised loss Lun, we use the commonly used standard flow loss... We ran the matrix factorization with 10 components and selected the five components that correlated most with the logit score (r is in the range [0.21, 0.34]). |