Accurate and Robust Feature Importance Estimation under Distribution Shifts
Authors: Jayaraman J. Thiagarajan, Vivek Narayanaswamy, Rushil Anirudh, Peer-Timo Bremer, Andreas Spanias7891-7898
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using empirical studies on several benchmark image and non-image data, we show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness. |
| Researcher Affiliation | Collaboration | 1Lawrence Livermore National Labs 2Arizona State University |
| Pseudocode | No | The paper describes algorithms and methods in detail but does not provide a clearly labeled pseudocode or algorithm block. |
| Open Source Code | No | The paper mentions links to third-party code (LIME, SHAP, CXPlain) but does not provide concrete access to the source code for PRo FILE, the methodology described in this paper. |
| Open Datasets | Yes | We consider a suite of synthetic and real-world datasets... (a) UCI Handwritten Digits dataset (Dua and Graff 2017), (b) Open ML benchmarks (Vanschoren et al. 2013), Kropt, Letter Image Recognition, Pokerhand and RBF datasets and (c) Cifar10 image classification dataset (Krizhevsky and Hinton 2009)... (b) Cifar10 to Cifar10C (Hendrycks and Dietterich 2019)... (c) MNIST-USPS: ...MNIST handwritten digits dataset (Le Cun, Cortes, and Burges 2010) and ...USPS dataset (Hull 1994). |
| Dataset Splits | Yes | For each of the UCI and Open ML datasets, we utilized 90% of the data while for Cifar10, we used the prescribed dataset of 50K RGB images of size 32 32 for training our proposed model... we used the held-out test set for our evaluation (90-10 split). |
| Hardware Specification | No | The paper mentions 'trained our proposed model' and 'networks were trained' but does not specify any particular hardware details such as GPU models, CPU types, or memory. |
| Software Dependencies | No | The paper mentions 'ADAM optimizer' and 'Re LU activations' but does not provide specific version numbers for any software dependencies or libraries used for the experiments. |
| Experiment Setup | Yes | For all non-imaging datasets, the black-box model was a 5 layer MLP with Re LU activations, each fully-connected (FC) layer in the loss estimator contained 16 units. In the case of Cifar-10, we used the standard Res Net-18 architecture, and the loss estimator used outputs from each residual blocks (with fully connected layers containing 128 hidden units). Finally, for the MNIST-USPS experiment, we used a 3 layer CNN with 2 FC layers. The loss estimator was designed to access outputs from the first 4 layers of the network and utilized FC layers with 16 units each. All networks were trained using the ADAM optimizer with a learning rate 0.001 and batch size 128. |