Attention-Based Pyramid Network for Segmentation and Classification of High-Resolution and Hyperspectral Remote Sensing Images
Unlike conventional natural (RGB) images, the inherent large scale and complex structures of remote sensing images pose major challenges such as spatial object distribution diversity and spectral information extraction when existing models are directly applied for image classification. In this study...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-10-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/12/21/3501 |
| _version_ | 1797549883796750336 |
|---|---|
| author | Qingsong Xu Xin Yuan Chaojun Ouyang Yue Zeng |
| author_facet | Qingsong Xu Xin Yuan Chaojun Ouyang Yue Zeng |
| author_sort | Qingsong Xu |
| collection | DOAJ |
| description | Unlike conventional natural (RGB) images, the inherent large scale and complex structures of remote sensing images pose major challenges such as spatial object distribution diversity and spectral information extraction when existing models are directly applied for image classification. In this study, we develop an attention-based pyramid network for segmentation and classification of remote sensing datasets. Attention mechanisms are used to develop the following modules: (<i>i</i>) a novel and robust attention-based <i>multi-scale fusion</i> method effectively fuses useful spatial or spectral information at different and same scales; (<inline-formula><math display="inline"><semantics><mrow><mi>i</mi><mi>i</mi></mrow></semantics></math></inline-formula>) a <i>region pyramid attention</i> mechanism using region-based attention addresses the target geometric size diversity in large-scale remote sensing images; and (<inline-formula><math display="inline"><semantics><mrow><mi>i</mi><mi>i</mi><mi>i</mi></mrow></semantics></math></inline-formula>) <i>cross-scale attention</i> in our adaptive atrous spatial pyramid pooling network adapts to varied contents in a feature-embedded space. Different forms of feature fusion pyramid frameworks are established by combining these attention-based modules. First, a novel segmentation framework, called the heavy-weight spatial feature fusion pyramid network (FFPNet), is proposed to address the spatial problem of high-resolution remote sensing images. Second, an end-to-end spatial-spectral FFPNet is presented for classifying hyperspectral images. Experiments conducted on ISPRS Vaihingen and ISPRS Potsdam high-resolution datasets demonstrate the competitive segmentation accuracy achieved by the proposed heavy-weight spatial FFPNet. Furthermore, experiments on the Indian Pines and the University of Pavia hyperspectral datasets indicate that the proposed spatial-spectral FFPNet outperforms the current state-of-the-art methods in hyperspectral image classification. |
| first_indexed | 2024-03-10T15:21:44Z |
| format | Article |
| id | doaj.art-71233185ee084ff6ab831bf6d6f9a5d6 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T15:21:44Z |
| publishDate | 2020-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-71233185ee084ff6ab831bf6d6f9a5d62023-11-20T18:26:40ZengMDPI AGRemote Sensing2072-42922020-10-011221350110.3390/rs12213501Attention-Based Pyramid Network for Segmentation and Classification of High-Resolution and Hyperspectral Remote Sensing ImagesQingsong Xu0Xin Yuan1Chaojun Ouyang2Yue Zeng3Key Laboratory of Mountain Hazards and Surface Process, Institute of Mountain Hazards and Environment, Chinese Academy of Sciences, Chengdu 610041, ChinaBell Labs, Murray Hill, NJ 07974, USAKey Laboratory of Mountain Hazards and Surface Process, Institute of Mountain Hazards and Environment, Chinese Academy of Sciences, Chengdu 610041, ChinaSchool of Economics and Management, Southwest Jiao Tong University, Chengdu 610031, ChinaUnlike conventional natural (RGB) images, the inherent large scale and complex structures of remote sensing images pose major challenges such as spatial object distribution diversity and spectral information extraction when existing models are directly applied for image classification. In this study, we develop an attention-based pyramid network for segmentation and classification of remote sensing datasets. Attention mechanisms are used to develop the following modules: (<i>i</i>) a novel and robust attention-based <i>multi-scale fusion</i> method effectively fuses useful spatial or spectral information at different and same scales; (<inline-formula><math display="inline"><semantics><mrow><mi>i</mi><mi>i</mi></mrow></semantics></math></inline-formula>) a <i>region pyramid attention</i> mechanism using region-based attention addresses the target geometric size diversity in large-scale remote sensing images; and (<inline-formula><math display="inline"><semantics><mrow><mi>i</mi><mi>i</mi><mi>i</mi></mrow></semantics></math></inline-formula>) <i>cross-scale attention</i> in our adaptive atrous spatial pyramid pooling network adapts to varied contents in a feature-embedded space. Different forms of feature fusion pyramid frameworks are established by combining these attention-based modules. First, a novel segmentation framework, called the heavy-weight spatial feature fusion pyramid network (FFPNet), is proposed to address the spatial problem of high-resolution remote sensing images. Second, an end-to-end spatial-spectral FFPNet is presented for classifying hyperspectral images. Experiments conducted on ISPRS Vaihingen and ISPRS Potsdam high-resolution datasets demonstrate the competitive segmentation accuracy achieved by the proposed heavy-weight spatial FFPNet. Furthermore, experiments on the Indian Pines and the University of Pavia hyperspectral datasets indicate that the proposed spatial-spectral FFPNet outperforms the current state-of-the-art methods in hyperspectral image classification.https://www.mdpi.com/2072-4292/12/21/3501high-resolution and hyperspectral imagesspatial object distribution diversityspectral information extractionattention-based pyramid networkheavy-weight spatial feature fusion pyramid network (FFPNet)spatial-spectral FFPNet |
| spellingShingle | Qingsong Xu Xin Yuan Chaojun Ouyang Yue Zeng Attention-Based Pyramid Network for Segmentation and Classification of High-Resolution and Hyperspectral Remote Sensing Images Remote Sensing high-resolution and hyperspectral images spatial object distribution diversity spectral information extraction attention-based pyramid network heavy-weight spatial feature fusion pyramid network (FFPNet) spatial-spectral FFPNet |
| title | Attention-Based Pyramid Network for Segmentation and Classification of High-Resolution and Hyperspectral Remote Sensing Images |
| title_full | Attention-Based Pyramid Network for Segmentation and Classification of High-Resolution and Hyperspectral Remote Sensing Images |
| title_fullStr | Attention-Based Pyramid Network for Segmentation and Classification of High-Resolution and Hyperspectral Remote Sensing Images |
| title_full_unstemmed | Attention-Based Pyramid Network for Segmentation and Classification of High-Resolution and Hyperspectral Remote Sensing Images |
| title_short | Attention-Based Pyramid Network for Segmentation and Classification of High-Resolution and Hyperspectral Remote Sensing Images |
| title_sort | attention based pyramid network for segmentation and classification of high resolution and hyperspectral remote sensing images |
| topic | high-resolution and hyperspectral images spatial object distribution diversity spectral information extraction attention-based pyramid network heavy-weight spatial feature fusion pyramid network (FFPNet) spatial-spectral FFPNet |
| url | https://www.mdpi.com/2072-4292/12/21/3501 |
| work_keys_str_mv | AT qingsongxu attentionbasedpyramidnetworkforsegmentationandclassificationofhighresolutionandhyperspectralremotesensingimages AT xinyuan attentionbasedpyramidnetworkforsegmentationandclassificationofhighresolutionandhyperspectralremotesensingimages AT chaojunouyang attentionbasedpyramidnetworkforsegmentationandclassificationofhighresolutionandhyperspectralremotesensingimages AT yuezeng attentionbasedpyramidnetworkforsegmentationandclassificationofhighresolutionandhyperspectralremotesensingimages |