Effect of Attention Mechanism in Deep Learning-Based Remote Sensing Image Processing: A Systematic Literature Review
Machine learning, particularly deep learning (DL), has become a central and state-of-the-art method for several computer vision applications and remote sensing (RS) image processing. Researchers are continually trying to improve the performance of the DL methods by developing new architectural desig...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/15/2965 |
| _version_ | 1797525169878597632 |
|---|---|
| author | Saman Ghaffarian João Valente Mariska van der Voort Bedir Tekinerdogan |
| author_facet | Saman Ghaffarian João Valente Mariska van der Voort Bedir Tekinerdogan |
| author_sort | Saman Ghaffarian |
| collection | DOAJ |
| description | Machine learning, particularly deep learning (DL), has become a central and state-of-the-art method for several computer vision applications and remote sensing (RS) image processing. Researchers are continually trying to improve the performance of the DL methods by developing new architectural designs of the networks and/or developing new techniques, such as attention mechanisms. Since the attention mechanism has been proposed, regardless of its type, it has been increasingly used for diverse RS applications to improve the performances of the existing DL methods. However, these methods are scattered over different studies impeding the selection and application of the feasible approaches. This study provides an overview of the developed attention mechanisms and how to integrate them with different deep learning neural network architectures. In addition, it aims to investigate the effect of the attention mechanism on deep learning-based RS image processing. We identified and analyzed the advances in the corresponding attention mechanism-based deep learning (At-DL) methods. A systematic literature review was performed to identify the trends in publications, publishers, improved DL methods, data types used, attention types used, overall accuracies achieved using At-DL methods, and extracted the current research directions, weaknesses, and open problems to provide insights and recommendations for future studies. For this, five main research questions were formulated to extract the required data and information from the literature. Furthermore, we categorized the papers regarding the addressed RS image processing tasks (e.g., image classification, object detection, and change detection) and discussed the results within each group. In total, 270 papers were retrieved, of which 176 papers were selected according to the defined exclusion criteria for further analysis and detailed review. The results reveal that most of the papers reported an increase in overall accuracy when using the attention mechanism within the DL methods for image classification, image segmentation, change detection, and object detection using remote sensing images. |
| first_indexed | 2024-03-10T09:08:59Z |
| format | Article |
| id | doaj.art-3caf232409fc46ea896621cb688921dc |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T09:08:59Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-3caf232409fc46ea896621cb688921dc2023-11-22T06:06:50ZengMDPI AGRemote Sensing2072-42922021-07-011315296510.3390/rs13152965Effect of Attention Mechanism in Deep Learning-Based Remote Sensing Image Processing: A Systematic Literature ReviewSaman Ghaffarian0João Valente1Mariska van der Voort2Bedir Tekinerdogan3Information Technology Group, Wageningen University & Research, 6707 KN Wageningen, The NetherlandsInformation Technology Group, Wageningen University & Research, 6707 KN Wageningen, The NetherlandsBusiness Economics Group, Wageningen University & Research, 6700 EW Wageningen, The NetherlandsInformation Technology Group, Wageningen University & Research, 6707 KN Wageningen, The NetherlandsMachine learning, particularly deep learning (DL), has become a central and state-of-the-art method for several computer vision applications and remote sensing (RS) image processing. Researchers are continually trying to improve the performance of the DL methods by developing new architectural designs of the networks and/or developing new techniques, such as attention mechanisms. Since the attention mechanism has been proposed, regardless of its type, it has been increasingly used for diverse RS applications to improve the performances of the existing DL methods. However, these methods are scattered over different studies impeding the selection and application of the feasible approaches. This study provides an overview of the developed attention mechanisms and how to integrate them with different deep learning neural network architectures. In addition, it aims to investigate the effect of the attention mechanism on deep learning-based RS image processing. We identified and analyzed the advances in the corresponding attention mechanism-based deep learning (At-DL) methods. A systematic literature review was performed to identify the trends in publications, publishers, improved DL methods, data types used, attention types used, overall accuracies achieved using At-DL methods, and extracted the current research directions, weaknesses, and open problems to provide insights and recommendations for future studies. For this, five main research questions were formulated to extract the required data and information from the literature. Furthermore, we categorized the papers regarding the addressed RS image processing tasks (e.g., image classification, object detection, and change detection) and discussed the results within each group. In total, 270 papers were retrieved, of which 176 papers were selected according to the defined exclusion criteria for further analysis and detailed review. The results reveal that most of the papers reported an increase in overall accuracy when using the attention mechanism within the DL methods for image classification, image segmentation, change detection, and object detection using remote sensing images.https://www.mdpi.com/2072-4292/13/15/2965remote sensingimage processingattention mechanismspatial attentionchannel attentiondeep learning |
| spellingShingle | Saman Ghaffarian João Valente Mariska van der Voort Bedir Tekinerdogan Effect of Attention Mechanism in Deep Learning-Based Remote Sensing Image Processing: A Systematic Literature Review Remote Sensing remote sensing image processing attention mechanism spatial attention channel attention deep learning |
| title | Effect of Attention Mechanism in Deep Learning-Based Remote Sensing Image Processing: A Systematic Literature Review |
| title_full | Effect of Attention Mechanism in Deep Learning-Based Remote Sensing Image Processing: A Systematic Literature Review |
| title_fullStr | Effect of Attention Mechanism in Deep Learning-Based Remote Sensing Image Processing: A Systematic Literature Review |
| title_full_unstemmed | Effect of Attention Mechanism in Deep Learning-Based Remote Sensing Image Processing: A Systematic Literature Review |
| title_short | Effect of Attention Mechanism in Deep Learning-Based Remote Sensing Image Processing: A Systematic Literature Review |
| title_sort | effect of attention mechanism in deep learning based remote sensing image processing a systematic literature review |
| topic | remote sensing image processing attention mechanism spatial attention channel attention deep learning |
| url | https://www.mdpi.com/2072-4292/13/15/2965 |
| work_keys_str_mv | AT samanghaffarian effectofattentionmechanismindeeplearningbasedremotesensingimageprocessingasystematicliteraturereview AT joaovalente effectofattentionmechanismindeeplearningbasedremotesensingimageprocessingasystematicliteraturereview AT mariskavandervoort effectofattentionmechanismindeeplearningbasedremotesensingimageprocessingasystematicliteraturereview AT bedirtekinerdogan effectofattentionmechanismindeeplearningbasedremotesensingimageprocessingasystematicliteraturereview |