Explaining image classifiers using statistical fault localization
The black-box nature of deep neural networks (DNNs) makes it impossible to understand why a particular output is produced, creating demand for “Explainable AI”. In this paper, we show that statistical fault localization (SFL) techniques from software engineering deliver high quality explanations of...
| Main Authors: | , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Springer
2020
|
| _version_ | 1797075887138537472 |
|---|---|
| author | Sun, Y Chockler, H Huang, X Kroening, D |
| author_facet | Sun, Y Chockler, H Huang, X Kroening, D |
| author_sort | Sun, Y |
| collection | OXFORD |
| description | The black-box nature of deep neural networks (DNNs) makes it impossible to understand why a particular output is produced, creating demand for “Explainable AI”. In this paper, we show that statistical fault localization (SFL) techniques from software engineering deliver high quality explanations of the outputs of DNNs, where we define an explanation as a minimal subset of features sufficient for making the same decision as for the original input. We present an algorithm and a tool called DeepCover, which synthesizes a ranking of the features of the inputs using SFL and constructs explanations for the decisions of the DNN based on this ranking. We compare explanations produced by DeepCover with those of the state-of-the-art tools gradcam, lime, shap, rise and extremal and show that explanations generated by DeepCover are consistently better across a broad set of experiments. On a benchmark set with known ground truth, DeepCover achieves 76.7% accuracy, which is 6% better than the second best extremal. |
| first_indexed | 2024-03-06T23:56:35Z |
| format | Conference item |
| id | oxford-uuid:746d153b-61c1-47c3-93f1-9ba34ff9c3e0 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T23:56:35Z |
| publishDate | 2020 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:746d153b-61c1-47c3-93f1-9ba34ff9c3e02022-03-26T20:02:49ZExplaining image classifiers using statistical fault localizationConference itemhttp://purl.org/coar/resource_type/c_5794uuid:746d153b-61c1-47c3-93f1-9ba34ff9c3e0EnglishSymplectic ElementsSpringer2020Sun, YChockler, HHuang, XKroening, DThe black-box nature of deep neural networks (DNNs) makes it impossible to understand why a particular output is produced, creating demand for “Explainable AI”. In this paper, we show that statistical fault localization (SFL) techniques from software engineering deliver high quality explanations of the outputs of DNNs, where we define an explanation as a minimal subset of features sufficient for making the same decision as for the original input. We present an algorithm and a tool called DeepCover, which synthesizes a ranking of the features of the inputs using SFL and constructs explanations for the decisions of the DNN based on this ranking. We compare explanations produced by DeepCover with those of the state-of-the-art tools gradcam, lime, shap, rise and extremal and show that explanations generated by DeepCover are consistently better across a broad set of experiments. On a benchmark set with known ground truth, DeepCover achieves 76.7% accuracy, which is 6% better than the second best extremal. |
| spellingShingle | Sun, Y Chockler, H Huang, X Kroening, D Explaining image classifiers using statistical fault localization |
| title | Explaining image classifiers using statistical fault localization |
| title_full | Explaining image classifiers using statistical fault localization |
| title_fullStr | Explaining image classifiers using statistical fault localization |
| title_full_unstemmed | Explaining image classifiers using statistical fault localization |
| title_short | Explaining image classifiers using statistical fault localization |
| title_sort | explaining image classifiers using statistical fault localization |
| work_keys_str_mv | AT suny explainingimageclassifiersusingstatisticalfaultlocalization AT chocklerh explainingimageclassifiersusingstatisticalfaultlocalization AT huangx explainingimageclassifiersusingstatisticalfaultlocalization AT kroeningd explainingimageclassifiersusingstatisticalfaultlocalization |