FMAM-Net: Fusion Multi-Scale Attention Mechanism Network for Building Segmentation in Remote Sensing Images
As the largest target in remote sensing images, buildings have important application value in urban planning and old city reconstruction. However, most networks have poor recognition ability on high resolution images, resulting in blurred boundaries in the segmented building maps. Then, the similari...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9996387/ |
| _version_ | 1797966788442456064 |
|---|---|
| author | Huanran Ye Run Zhou Jianhao Wang Zhiliang Huang |
| author_facet | Huanran Ye Run Zhou Jianhao Wang Zhiliang Huang |
| author_sort | Huanran Ye |
| collection | DOAJ |
| description | As the largest target in remote sensing images, buildings have important application value in urban planning and old city reconstruction. However, most networks have poor recognition ability on high resolution images, resulting in blurred boundaries in the segmented building maps. Then, the similarity between buildings and backgrounds will lead to inter-class indistinction. Finally, the diversity of buildings brings difficulties to segmentation, which requires the network to have better generalization ability. To address these problems, we propose Fusion Multi-scale Attention Mechanism Network (FMAM-Net). Firstly, we design Feature Refine Compensation Module(FRCM) to improve the boundary ambiguity problem, including Feature Refinement Module(FRM) and Feature Compensation Module(FCM). FRM utilizes the densely connected architecture to refine features and increase recognition capabilities. FCM introduces low-level features to make up for the lack of boundary information in high-level features. Secondly, to handle inter-class indistinction, we design Tandem Attention Module(TAM) and Parallel Attention Module(PAM). TAM is designed to sequentially filter some features from channels and spaces for adaptive feature refinement. PAM combines context information and uses high-level features to guide low-level features to select more distinguishable features. Finally, based on the binary cross entropy loss function, we add an evaluation index to reduce the error caused by determining the optimization direction only through cross entropy. On the Inria Aerial Image Labeling Dataset, FMAM-Net achieves mean IoU of 85.34%, which is 5.58% higher than AMUNet and 3.77% higher than our baseline(U-Net ResNet-34). On the WHU Dataset, IoU reached the maximum value of 91.06% on FMAM-Net, 1.67% higher than SARB-UNet and 0.2% higher than MAP-Net. The visualization results show that FMAM-Net improves the fuzzy boundary of building segmentation and reduces the inter-class indistinction. |
| first_indexed | 2024-04-11T02:20:53Z |
| format | Article |
| id | doaj.art-c7b36fbf18474251b6f05cf65eed91c7 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T02:20:53Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c7b36fbf18474251b6f05cf65eed91c72023-01-03T00:00:11ZengIEEEIEEE Access2169-35362022-01-011013424113425110.1109/ACCESS.2022.32313629996387FMAM-Net: Fusion Multi-Scale Attention Mechanism Network for Building Segmentation in Remote Sensing ImagesHuanran Ye0https://orcid.org/0000-0001-6658-0368Run Zhou1Jianhao Wang2https://orcid.org/0000-0002-9782-3243Zhiliang Huang3https://orcid.org/0000-0003-0876-063XSchool of Mechanical and Electrical Information, Yiwu Industrial & Commercial College, Jinhua, ChinaSchool of Mechanical and Electrical Information, Yiwu Industrial & Commercial College, Jinhua, ChinaSchool of Mechanical and Electrical Information, Yiwu Industrial & Commercial College, Jinhua, ChinaCollege of Physics and Electronic Information Engineering, Zhejiang Normal University, Jinhua, ChinaAs the largest target in remote sensing images, buildings have important application value in urban planning and old city reconstruction. However, most networks have poor recognition ability on high resolution images, resulting in blurred boundaries in the segmented building maps. Then, the similarity between buildings and backgrounds will lead to inter-class indistinction. Finally, the diversity of buildings brings difficulties to segmentation, which requires the network to have better generalization ability. To address these problems, we propose Fusion Multi-scale Attention Mechanism Network (FMAM-Net). Firstly, we design Feature Refine Compensation Module(FRCM) to improve the boundary ambiguity problem, including Feature Refinement Module(FRM) and Feature Compensation Module(FCM). FRM utilizes the densely connected architecture to refine features and increase recognition capabilities. FCM introduces low-level features to make up for the lack of boundary information in high-level features. Secondly, to handle inter-class indistinction, we design Tandem Attention Module(TAM) and Parallel Attention Module(PAM). TAM is designed to sequentially filter some features from channels and spaces for adaptive feature refinement. PAM combines context information and uses high-level features to guide low-level features to select more distinguishable features. Finally, based on the binary cross entropy loss function, we add an evaluation index to reduce the error caused by determining the optimization direction only through cross entropy. On the Inria Aerial Image Labeling Dataset, FMAM-Net achieves mean IoU of 85.34%, which is 5.58% higher than AMUNet and 3.77% higher than our baseline(U-Net ResNet-34). On the WHU Dataset, IoU reached the maximum value of 91.06% on FMAM-Net, 1.67% higher than SARB-UNet and 0.2% higher than MAP-Net. The visualization results show that FMAM-Net improves the fuzzy boundary of building segmentation and reduces the inter-class indistinction.https://ieeexplore.ieee.org/document/9996387/Remote sensing imagebuilding segmentationattention mechanismfeature refinementencoder-decoder |
| spellingShingle | Huanran Ye Run Zhou Jianhao Wang Zhiliang Huang FMAM-Net: Fusion Multi-Scale Attention Mechanism Network for Building Segmentation in Remote Sensing Images IEEE Access Remote sensing image building segmentation attention mechanism feature refinement encoder-decoder |
| title | FMAM-Net: Fusion Multi-Scale Attention Mechanism Network for Building Segmentation in Remote Sensing Images |
| title_full | FMAM-Net: Fusion Multi-Scale Attention Mechanism Network for Building Segmentation in Remote Sensing Images |
| title_fullStr | FMAM-Net: Fusion Multi-Scale Attention Mechanism Network for Building Segmentation in Remote Sensing Images |
| title_full_unstemmed | FMAM-Net: Fusion Multi-Scale Attention Mechanism Network for Building Segmentation in Remote Sensing Images |
| title_short | FMAM-Net: Fusion Multi-Scale Attention Mechanism Network for Building Segmentation in Remote Sensing Images |
| title_sort | fmam net fusion multi scale attention mechanism network for building segmentation in remote sensing images |
| topic | Remote sensing image building segmentation attention mechanism feature refinement encoder-decoder |
| url | https://ieeexplore.ieee.org/document/9996387/ |
| work_keys_str_mv | AT huanranye fmamnetfusionmultiscaleattentionmechanismnetworkforbuildingsegmentationinremotesensingimages AT runzhou fmamnetfusionmultiscaleattentionmechanismnetworkforbuildingsegmentationinremotesensingimages AT jianhaowang fmamnetfusionmultiscaleattentionmechanismnetworkforbuildingsegmentationinremotesensingimages AT zhilianghuang fmamnetfusionmultiscaleattentionmechanismnetworkforbuildingsegmentationinremotesensingimages |