An interpretable approach for automatic aesthetic assessment of remote sensing images
The increase of remote sensing images in recent decades has resulted in their use in non-scientific fields such as environmental protection, education, and art. In this situation, we need to focus on the aesthetic assessment of remote sensing, which has received little attention in research. While a...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-11-01
|
| Series: | Frontiers in Computational Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fncom.2022.1077439/full |
| _version_ | 1811305302962733056 |
|---|---|
| author | Jingru Tong Guo Zhang Peijie Kong Yu Rao Zhengkai Wei Hao Cui Qing Guan |
| author_facet | Jingru Tong Guo Zhang Peijie Kong Yu Rao Zhengkai Wei Hao Cui Qing Guan |
| author_sort | Jingru Tong |
| collection | DOAJ |
| description | The increase of remote sensing images in recent decades has resulted in their use in non-scientific fields such as environmental protection, education, and art. In this situation, we need to focus on the aesthetic assessment of remote sensing, which has received little attention in research. While according to studies on human brain’s attention mechanism, certain areas of an image can trigger visual stimuli during aesthetic evaluation. Inspired by this, we used convolutional neural network (CNN), a deep learning model resembling the human neural system, for image aesthetic assessment. So we propose an interpretable approach for automatic aesthetic assessment of remote sensing images. Firstly, we created the Remote Sensing Aesthetics Dataset (RSAD). We collected remote sensing images from Google Earth, designed the four evaluation criteria of remote sensing image aesthetic quality—color harmony, light and shadow, prominent theme, and visual balance—and then labeled the samples based on expert photographers’ judgment on the four evaluation criteria. Secondly, we feed RSAD into the ResNet-18 architecture for training. Experimental results show that the proposed method can accurately identify visually pleasing remote sensing images. Finally, we provided a visual explanation of aesthetic assessment by adopting Gradient-weighted Class Activation Mapping (Grad-CAM) to highlight the important image area that influenced model’s decision. Overall, this paper is the first to propose and realize automatic aesthetic assessment of remote sensing images, contributing to the non-scientific applications of remote sensing and demonstrating the interpretability of deep-learning based image aesthetic evaluation. |
| first_indexed | 2024-04-13T08:23:01Z |
| format | Article |
| id | doaj.art-fa4476f7da2d48aa8277484f44342c5a |
| institution | Directory Open Access Journal |
| issn | 1662-5188 |
| language | English |
| last_indexed | 2024-04-13T08:23:01Z |
| publishDate | 2022-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computational Neuroscience |
| spelling | doaj.art-fa4476f7da2d48aa8277484f44342c5a2022-12-22T02:54:35ZengFrontiers Media S.A.Frontiers in Computational Neuroscience1662-51882022-11-011610.3389/fncom.2022.10774391077439An interpretable approach for automatic aesthetic assessment of remote sensing imagesJingru Tong0Guo Zhang1Peijie Kong2Yu Rao3Zhengkai Wei4Hao Cui5Qing Guan6School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaState Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaState Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, ChinaState Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, ChinaThe increase of remote sensing images in recent decades has resulted in their use in non-scientific fields such as environmental protection, education, and art. In this situation, we need to focus on the aesthetic assessment of remote sensing, which has received little attention in research. While according to studies on human brain’s attention mechanism, certain areas of an image can trigger visual stimuli during aesthetic evaluation. Inspired by this, we used convolutional neural network (CNN), a deep learning model resembling the human neural system, for image aesthetic assessment. So we propose an interpretable approach for automatic aesthetic assessment of remote sensing images. Firstly, we created the Remote Sensing Aesthetics Dataset (RSAD). We collected remote sensing images from Google Earth, designed the four evaluation criteria of remote sensing image aesthetic quality—color harmony, light and shadow, prominent theme, and visual balance—and then labeled the samples based on expert photographers’ judgment on the four evaluation criteria. Secondly, we feed RSAD into the ResNet-18 architecture for training. Experimental results show that the proposed method can accurately identify visually pleasing remote sensing images. Finally, we provided a visual explanation of aesthetic assessment by adopting Gradient-weighted Class Activation Mapping (Grad-CAM) to highlight the important image area that influenced model’s decision. Overall, this paper is the first to propose and realize automatic aesthetic assessment of remote sensing images, contributing to the non-scientific applications of remote sensing and demonstrating the interpretability of deep-learning based image aesthetic evaluation.https://www.frontiersin.org/articles/10.3389/fncom.2022.1077439/fullremote sensing imagesaesthetic assessmentaesthetic qualityinterpretabilityattention mechanismdeep learning |
| spellingShingle | Jingru Tong Guo Zhang Peijie Kong Yu Rao Zhengkai Wei Hao Cui Qing Guan An interpretable approach for automatic aesthetic assessment of remote sensing images Frontiers in Computational Neuroscience remote sensing images aesthetic assessment aesthetic quality interpretability attention mechanism deep learning |
| title | An interpretable approach for automatic aesthetic assessment of remote sensing images |
| title_full | An interpretable approach for automatic aesthetic assessment of remote sensing images |
| title_fullStr | An interpretable approach for automatic aesthetic assessment of remote sensing images |
| title_full_unstemmed | An interpretable approach for automatic aesthetic assessment of remote sensing images |
| title_short | An interpretable approach for automatic aesthetic assessment of remote sensing images |
| title_sort | interpretable approach for automatic aesthetic assessment of remote sensing images |
| topic | remote sensing images aesthetic assessment aesthetic quality interpretability attention mechanism deep learning |
| url | https://www.frontiersin.org/articles/10.3389/fncom.2022.1077439/full |
| work_keys_str_mv | AT jingrutong aninterpretableapproachforautomaticaestheticassessmentofremotesensingimages AT guozhang aninterpretableapproachforautomaticaestheticassessmentofremotesensingimages AT peijiekong aninterpretableapproachforautomaticaestheticassessmentofremotesensingimages AT yurao aninterpretableapproachforautomaticaestheticassessmentofremotesensingimages AT zhengkaiwei aninterpretableapproachforautomaticaestheticassessmentofremotesensingimages AT haocui aninterpretableapproachforautomaticaestheticassessmentofremotesensingimages AT qingguan aninterpretableapproachforautomaticaestheticassessmentofremotesensingimages AT jingrutong interpretableapproachforautomaticaestheticassessmentofremotesensingimages AT guozhang interpretableapproachforautomaticaestheticassessmentofremotesensingimages AT peijiekong interpretableapproachforautomaticaestheticassessmentofremotesensingimages AT yurao interpretableapproachforautomaticaestheticassessmentofremotesensingimages AT zhengkaiwei interpretableapproachforautomaticaestheticassessmentofremotesensingimages AT haocui interpretableapproachforautomaticaestheticassessmentofremotesensingimages AT qingguan interpretableapproachforautomaticaestheticassessmentofremotesensingimages |