Chain Graph Explanation of Neural Network Based on Feature-Level Class Confusion
Despite increasing interest in developing interpretable machine learning methods, most recent studies have provided explanations only for single instances, require additional datasets, and are sensitive to hyperparameters. This paper proposes a confusion graph that reveals model weaknesses by constr...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-01-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/3/1523 |
| _version_ | 1797489088344883200 |
|---|---|
| author | Hyekyoung Hwang Eunbyung Park Jitae Shin |
| author_facet | Hyekyoung Hwang Eunbyung Park Jitae Shin |
| author_sort | Hyekyoung Hwang |
| collection | DOAJ |
| description | Despite increasing interest in developing interpretable machine learning methods, most recent studies have provided explanations only for single instances, require additional datasets, and are sensitive to hyperparameters. This paper proposes a confusion graph that reveals model weaknesses by constructing a confusion dictionary. Unlike other methods, which focus on the performance variation caused by single-neuron suppression, it defines the role of each neuron in two different perspectives: ‘correction’ and ‘violation’. Furthermore, our method can identify the class relationships in similar positions at the feature level, which can suggest improvements to the model. Finally, the proposed graph construction is model-agnostic and does not require additional data or tedious hyperparameter tuning. Experimental results show that the information loss from omitting the channels guided by the proposed graph can result in huge performance degradation, from 91% to 33%, while the proposed graph only retains 1% of total neurons. |
| first_indexed | 2024-03-10T00:11:31Z |
| format | Article |
| id | doaj.art-d3bd8933f35140b3b2ddf286a1194a4a |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T00:11:31Z |
| publishDate | 2022-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-d3bd8933f35140b3b2ddf286a1194a4a2023-11-23T15:59:00ZengMDPI AGApplied Sciences2076-34172022-01-01123152310.3390/app12031523Chain Graph Explanation of Neural Network Based on Feature-Level Class ConfusionHyekyoung Hwang0Eunbyung Park1Jitae Shin2Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, KoreaDepartment of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, KoreaDepartment of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, KoreaDespite increasing interest in developing interpretable machine learning methods, most recent studies have provided explanations only for single instances, require additional datasets, and are sensitive to hyperparameters. This paper proposes a confusion graph that reveals model weaknesses by constructing a confusion dictionary. Unlike other methods, which focus on the performance variation caused by single-neuron suppression, it defines the role of each neuron in two different perspectives: ‘correction’ and ‘violation’. Furthermore, our method can identify the class relationships in similar positions at the feature level, which can suggest improvements to the model. Finally, the proposed graph construction is model-agnostic and does not require additional data or tedious hyperparameter tuning. Experimental results show that the information loss from omitting the channels guided by the proposed graph can result in huge performance degradation, from 91% to 33%, while the proposed graph only retains 1% of total neurons.https://www.mdpi.com/2076-3417/12/3/1523computer visiondeep learningconvolution neural networkexplainable AI |
| spellingShingle | Hyekyoung Hwang Eunbyung Park Jitae Shin Chain Graph Explanation of Neural Network Based on Feature-Level Class Confusion Applied Sciences computer vision deep learning convolution neural network explainable AI |
| title | Chain Graph Explanation of Neural Network Based on Feature-Level Class Confusion |
| title_full | Chain Graph Explanation of Neural Network Based on Feature-Level Class Confusion |
| title_fullStr | Chain Graph Explanation of Neural Network Based on Feature-Level Class Confusion |
| title_full_unstemmed | Chain Graph Explanation of Neural Network Based on Feature-Level Class Confusion |
| title_short | Chain Graph Explanation of Neural Network Based on Feature-Level Class Confusion |
| title_sort | chain graph explanation of neural network based on feature level class confusion |
| topic | computer vision deep learning convolution neural network explainable AI |
| url | https://www.mdpi.com/2076-3417/12/3/1523 |
| work_keys_str_mv | AT hyekyounghwang chaingraphexplanationofneuralnetworkbasedonfeaturelevelclassconfusion AT eunbyungpark chaingraphexplanationofneuralnetworkbasedonfeaturelevelclassconfusion AT jitaeshin chaingraphexplanationofneuralnetworkbasedonfeaturelevelclassconfusion |