Prob-POS: A Framework for Improving Visual Explanations from Convolutional Neural Networks for Remote Sensing Image Classification
During the past decades, convolutional neural network (CNN)-based models have achieved notable success in remote sensing image classification due to their powerful feature representation ability. However, the lack of explainability during the decision-making process is a common criticism of these hi...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/13/3042 |
| _version_ | 1797408216552833024 |
|---|---|
| author | Xianpeng Guo Biao Hou Zitong Wu Bo Ren Shuang Wang Licheng Jiao |
| author_facet | Xianpeng Guo Biao Hou Zitong Wu Bo Ren Shuang Wang Licheng Jiao |
| author_sort | Xianpeng Guo |
| collection | DOAJ |
| description | During the past decades, convolutional neural network (CNN)-based models have achieved notable success in remote sensing image classification due to their powerful feature representation ability. However, the lack of explainability during the decision-making process is a common criticism of these high-capacity networks. Local explanation methods that provide visual saliency maps have attracted increasing attention as a means to surmount the barrier of explainability. However, the vast majority of research is conducted on the last convolutional layer, where the salient regions are unintelligible for partial remote sensing images, especially scenes that contain plentiful small targets or are similar to the texture image. To address these issues, we propose a novel framework called Prob-POS, which consists of the class-activation map based on the probe network (Prob-CAM) and the weighted probability of occlusion (wPO) selection strategy. The proposed probe network is a simple but effective architecture to generate elaborate explanation maps and can be applied to any layer of CNNs. The wPO is a quantified metric to evaluate the explanation effectiveness of each layer for different categories to automatically pick out the optimal explanation layer. Variational weights are taken into account to highlight the high-scoring regions in the explanation map. Experimental results on two publicly available datasets and three prevalent networks demonstrate that Prob-POS improves the faithfulness and explainability of CNNs on remote sensing images. |
| first_indexed | 2024-03-09T03:56:10Z |
| format | Article |
| id | doaj.art-09f7d702f38f46e5ac21abe1f827e316 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T03:56:10Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-09f7d702f38f46e5ac21abe1f827e3162023-12-03T14:20:08ZengMDPI AGRemote Sensing2072-42922022-06-011413304210.3390/rs14133042Prob-POS: A Framework for Improving Visual Explanations from Convolutional Neural Networks for Remote Sensing Image ClassificationXianpeng Guo0Biao Hou1Zitong Wu2Bo Ren3Shuang Wang4Licheng Jiao5The Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, Xi’an 710071, ChinaThe Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, Xi’an 710071, ChinaThe Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, Xi’an 710071, ChinaThe Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, Xi’an 710071, ChinaThe Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, Xi’an 710071, ChinaThe Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, Xi’an 710071, ChinaDuring the past decades, convolutional neural network (CNN)-based models have achieved notable success in remote sensing image classification due to their powerful feature representation ability. However, the lack of explainability during the decision-making process is a common criticism of these high-capacity networks. Local explanation methods that provide visual saliency maps have attracted increasing attention as a means to surmount the barrier of explainability. However, the vast majority of research is conducted on the last convolutional layer, where the salient regions are unintelligible for partial remote sensing images, especially scenes that contain plentiful small targets or are similar to the texture image. To address these issues, we propose a novel framework called Prob-POS, which consists of the class-activation map based on the probe network (Prob-CAM) and the weighted probability of occlusion (wPO) selection strategy. The proposed probe network is a simple but effective architecture to generate elaborate explanation maps and can be applied to any layer of CNNs. The wPO is a quantified metric to evaluate the explanation effectiveness of each layer for different categories to automatically pick out the optimal explanation layer. Variational weights are taken into account to highlight the high-scoring regions in the explanation map. Experimental results on two publicly available datasets and three prevalent networks demonstrate that Prob-POS improves the faithfulness and explainability of CNNs on remote sensing images.https://www.mdpi.com/2072-4292/14/13/3042convolutional neural networks (CNNs)visual explanationremote sensing image |
| spellingShingle | Xianpeng Guo Biao Hou Zitong Wu Bo Ren Shuang Wang Licheng Jiao Prob-POS: A Framework for Improving Visual Explanations from Convolutional Neural Networks for Remote Sensing Image Classification Remote Sensing convolutional neural networks (CNNs) visual explanation remote sensing image |
| title | Prob-POS: A Framework for Improving Visual Explanations from Convolutional Neural Networks for Remote Sensing Image Classification |
| title_full | Prob-POS: A Framework for Improving Visual Explanations from Convolutional Neural Networks for Remote Sensing Image Classification |
| title_fullStr | Prob-POS: A Framework for Improving Visual Explanations from Convolutional Neural Networks for Remote Sensing Image Classification |
| title_full_unstemmed | Prob-POS: A Framework for Improving Visual Explanations from Convolutional Neural Networks for Remote Sensing Image Classification |
| title_short | Prob-POS: A Framework for Improving Visual Explanations from Convolutional Neural Networks for Remote Sensing Image Classification |
| title_sort | prob pos a framework for improving visual explanations from convolutional neural networks for remote sensing image classification |
| topic | convolutional neural networks (CNNs) visual explanation remote sensing image |
| url | https://www.mdpi.com/2072-4292/14/13/3042 |
| work_keys_str_mv | AT xianpengguo probposaframeworkforimprovingvisualexplanationsfromconvolutionalneuralnetworksforremotesensingimageclassification AT biaohou probposaframeworkforimprovingvisualexplanationsfromconvolutionalneuralnetworksforremotesensingimageclassification AT zitongwu probposaframeworkforimprovingvisualexplanationsfromconvolutionalneuralnetworksforremotesensingimageclassification AT boren probposaframeworkforimprovingvisualexplanationsfromconvolutionalneuralnetworksforremotesensingimageclassification AT shuangwang probposaframeworkforimprovingvisualexplanationsfromconvolutionalneuralnetworksforremotesensingimageclassification AT lichengjiao probposaframeworkforimprovingvisualexplanationsfromconvolutionalneuralnetworksforremotesensingimageclassification |