MAE-BG: dual-stream boundary optimization for remote sensing image semantic segmentation
Deep learning has achieved remarkable performance in semantically segmenting remotely sensed images. However, the high-frequency detail loss caused by continuous convolution and pooling operations and the uncertainty introduced when annotating low-contrast objects with weak boundaries induce blurred...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2023-12-01
|
| Series: | Geocarto International |
| Subjects: | |
| Online Access: | http://dx.doi.org/10.1080/10106049.2023.2190622 |
| _version_ | 1811153891419488256 |
|---|---|
| author | Ruiqi Yang Chen Zheng Leiguang Wang Yili Zhao Zhitao Fu Qinling Dai |
| author_facet | Ruiqi Yang Chen Zheng Leiguang Wang Yili Zhao Zhitao Fu Qinling Dai |
| author_sort | Ruiqi Yang |
| collection | DOAJ |
| description | Deep learning has achieved remarkable performance in semantically segmenting remotely sensed images. However, the high-frequency detail loss caused by continuous convolution and pooling operations and the uncertainty introduced when annotating low-contrast objects with weak boundaries induce blurred object boundaries. Therefore, a dual-stream network MAE-BG, consisting of an edge detection (ED) branch and a smooth branch with boundary guidance (BG), is proposed. The ED branch is designed to enhance the weak edges that need to be preserved, simultaneously suppressing false responses caused by local texture. This mechanism is achieved by introducing improved multiple-attention edge detection blocks (MAE). Furthermore, two specific ED branches with MAE are designed to combine with typical deep convolutional (DC) and Codec infrastructures and result in two configurations of MAE-A and MAE-B. Meanwhile, multiscale edge information extracted by MAE networks is fed into the backbone networks to complement the detail loss caused by convolution and pooling operations. This results in smooth networks with BG. After that, the segmentation results with improved boundaries are obtained by stacking the output of the ED and smooth branches. The proposed algorithms were evaluated on the ISPRS Potsdam and Inria Aerial Image Labelling datasets. Comprehensive experiments show that the proposed method can precisely locate object boundaries and improve segmentation performance. The MAE-A branch leads to an overall accuracy (OA) of 89.16%, a mean intersection over union (MIOU) of 80.25% for Potsdam, and an OA of 96.61% and MIOU of 86.63% for Inria. Compared with the results without the proposed edge optimization blocks, the OAs from the Potsdam and Inria datasets increase by 5.49% and 7.64%, respectively. |
| first_indexed | 2024-03-11T23:46:52Z |
| format | Article |
| id | doaj.art-37ae02527f344a61ada358641774baee |
| institution | Directory Open Access Journal |
| issn | 1010-6049 1752-0762 |
| language | English |
| last_indexed | 2024-03-11T23:46:52Z |
| publishDate | 2023-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Geocarto International |
| spelling | doaj.art-37ae02527f344a61ada358641774baee2023-09-19T09:13:18ZengTaylor & Francis GroupGeocarto International1010-60491752-07622023-12-0138110.1080/10106049.2023.21906222190622MAE-BG: dual-stream boundary optimization for remote sensing image semantic segmentationRuiqi Yang0Chen Zheng1Leiguang Wang2Yili Zhao3Zhitao Fu4Qinling Dai5Institute of Big Data and Artificial Intelligence, Southwest Forestry UniversitySchool of Mathematics and Statistics, Henan UniversityInstitute of Big Data and Artificial Intelligence, Southwest Forestry UniversityInstitute of Big Data and Artificial Intelligence, Southwest Forestry UniversityFaculty of Land and Resources Engineering, Kunming University of Science and TechnologyArt and Design College, Southwest Forestry UniversityDeep learning has achieved remarkable performance in semantically segmenting remotely sensed images. However, the high-frequency detail loss caused by continuous convolution and pooling operations and the uncertainty introduced when annotating low-contrast objects with weak boundaries induce blurred object boundaries. Therefore, a dual-stream network MAE-BG, consisting of an edge detection (ED) branch and a smooth branch with boundary guidance (BG), is proposed. The ED branch is designed to enhance the weak edges that need to be preserved, simultaneously suppressing false responses caused by local texture. This mechanism is achieved by introducing improved multiple-attention edge detection blocks (MAE). Furthermore, two specific ED branches with MAE are designed to combine with typical deep convolutional (DC) and Codec infrastructures and result in two configurations of MAE-A and MAE-B. Meanwhile, multiscale edge information extracted by MAE networks is fed into the backbone networks to complement the detail loss caused by convolution and pooling operations. This results in smooth networks with BG. After that, the segmentation results with improved boundaries are obtained by stacking the output of the ED and smooth branches. The proposed algorithms were evaluated on the ISPRS Potsdam and Inria Aerial Image Labelling datasets. Comprehensive experiments show that the proposed method can precisely locate object boundaries and improve segmentation performance. The MAE-A branch leads to an overall accuracy (OA) of 89.16%, a mean intersection over union (MIOU) of 80.25% for Potsdam, and an OA of 96.61% and MIOU of 86.63% for Inria. Compared with the results without the proposed edge optimization blocks, the OAs from the Potsdam and Inria datasets increase by 5.49% and 7.64%, respectively.http://dx.doi.org/10.1080/10106049.2023.2190622deep learningboundary optimizationsqueeze and excitationsemantic segmentationremote sensing |
| spellingShingle | Ruiqi Yang Chen Zheng Leiguang Wang Yili Zhao Zhitao Fu Qinling Dai MAE-BG: dual-stream boundary optimization for remote sensing image semantic segmentation Geocarto International deep learning boundary optimization squeeze and excitation semantic segmentation remote sensing |
| title | MAE-BG: dual-stream boundary optimization for remote sensing image semantic segmentation |
| title_full | MAE-BG: dual-stream boundary optimization for remote sensing image semantic segmentation |
| title_fullStr | MAE-BG: dual-stream boundary optimization for remote sensing image semantic segmentation |
| title_full_unstemmed | MAE-BG: dual-stream boundary optimization for remote sensing image semantic segmentation |
| title_short | MAE-BG: dual-stream boundary optimization for remote sensing image semantic segmentation |
| title_sort | mae bg dual stream boundary optimization for remote sensing image semantic segmentation |
| topic | deep learning boundary optimization squeeze and excitation semantic segmentation remote sensing |
| url | http://dx.doi.org/10.1080/10106049.2023.2190622 |
| work_keys_str_mv | AT ruiqiyang maebgdualstreamboundaryoptimizationforremotesensingimagesemanticsegmentation AT chenzheng maebgdualstreamboundaryoptimizationforremotesensingimagesemanticsegmentation AT leiguangwang maebgdualstreamboundaryoptimizationforremotesensingimagesemanticsegmentation AT yilizhao maebgdualstreamboundaryoptimizationforremotesensingimagesemanticsegmentation AT zhitaofu maebgdualstreamboundaryoptimizationforremotesensingimagesemanticsegmentation AT qinlingdai maebgdualstreamboundaryoptimizationforremotesensingimagesemanticsegmentation |