Attribution-Based Confidence Metric For Deep Neural Networks
Authors: Susmit Jha, Sunny Raj, Steven Fernandes, Sumit K. Jha, Somesh Jha, Brian Jalaian, Gunjan Verma, Ananthram Swami
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We mathematically motivate the proposed metric and evaluate its effectiveness with two sets of experiments. First, we study the change in accuracy and the associated confidence over out-of-distribution inputs. Second, we consider several digital and physically realizable attacks such as FGSM, CW, Deep Fool, PGD, and adversarial patch generation methods. The ABC metric is low on out-of-distribution data and adversarial examples, where the accuracy of the model is also low. These experiments demonstrate the effectiveness of the ABC metric towards creating more trustworthy and resilient DNNs. |
| Researcher Affiliation | Collaboration | Susmit Jha Computer Science Laboratory SRI International Sunny Raj, Steven Lawrence Fernandes, Sumit Kumar Jha Computer Science Department University of Central Florida, Orlando Somesh Jha University of Wisconsin-Madison and Xaipient Brian Jalaian, Gunjan Verma, Ananthram Swami US Army Research Laboratory Adelphi |
| Pseudocode | Yes | Algorithm 1 Evaluate ABC confidence metric c(F, x) of machine learning model F on input x Input: Model F, Input x with features x1, x2, . . . xn, Sample size S Output: ABC metric c(F, x) 1: A1, . . . An Attributions of features x1, x2, . . . xn from input x 2: i F(x) {Obtain model prediction} 3: for j = 1 to n do 4: P(xj) |Aj/xj| Pn k=1 |Ak/xk| 5: end for 6: Generate S samples by mutating feature xj of input x to baseline xb j with probability P(xj) 7: Obtain the output of the model on the S samples. 8: c(F, x) Sconform/S where model s output on Sconform samples is i 9: return c(F, x) as confidence metric (ABC) of prediction by the model F on the input x |
| Open Source Code | No | The paper does not provide an explicit statement or link to the open-source code for the proposed ABC metric or any other methodology described. |
| Open Datasets | Yes | We empirically evaluate the ABC metric over MNIST and Image Net datasets using (a) out-of-distribution data, (b) adversarial inputs generated using digital attacks such as FGSM, PGD, CW and Deep Fool, and (c) physically-realizable adversarial patches and La VAN attacks. Out-of-distribution data: MNIST [65] with rotation and background, not MNIST [66] and Fashion MNIST [67]. |
| Dataset Splits | No | The paper mentions 'held-out validation set' in the context of related work on calibration models but does not provide specific train/validation/test dataset splits for its own experiments. |
| Hardware Specification | Yes | All experiments were conducted on a 8 core Intel Core i9-9900K 3.60GHz CPU with NVIDIA Titan RTX Graphics and 32 GB RAM. |
| Software Dependencies | No | The paper mentions using 'Cleverhans' but does not specify any version numbers for this or other software dependencies. |
| Experiment Setup | No | The paper does not provide specific experimental setup details such as hyperparameter values (e.g., learning rate, batch size, number of epochs), optimizer settings, or other system-level training configurations. |