IMPROVING SEMANTIC SEGMENTATION PERFORMANCE BY JOINTLY USING HIGH RESOLUTION REMOTE SENSING IMAGE AND NDSM
Semantic segmentation algorithms based on full convolutional neural network have greatly improved segmentation accuracy of high-resolution remote sensing (RS) images. However, the interpretation of RS images from single sensor is still challenging due to the variety and complexity of land objects, t...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2022-05-01
|
| Series: | ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-3-2022/77/2022/isprs-annals-V-3-2022-77-2022.pdf |
| _version_ | 1818203000032722944 |
|---|---|
| author | R. Yang Q. Dai H. Cheng Y. Zhang N. Chen L. Wang |
| author_facet | R. Yang Q. Dai H. Cheng Y. Zhang N. Chen L. Wang |
| author_sort | R. Yang |
| collection | DOAJ |
| description | Semantic segmentation algorithms based on full convolutional neural network have greatly improved segmentation accuracy of high-resolution remote sensing (RS) images. However, the interpretation of RS images from single sensor is still challenging due to the variety and complexity of land objects, the extreme imbalance distributions of land objects on size and numbers. In contrast, multiple sensors can provide complementary information on the land classes, and thus benefit the interpretation. In this context, this research explores the joint use of RGB optical bands and normalized DSM (nDSM) to analyze an urban scene. The method firstly concatenated three channels RGB image and one channel nDSM band into a four-channel image. Thereafter, ResNet-101 network with fine adjustment were utilized as the backbone network to retain multiple feature information by residual blocks. Then the augmented RGB and nDSM images were used to training the network. The established model was evaluated on the Postdam test set. Results show that the proposed method achieves 86.85% on Overall Accuracy (OA), 77.42% Mean Intersection Over Union (MIOU), which is 6.88% and 11.39% higher than the result achieved by single RGB images. Especially, small targets, such as car and tree, are higher. The experimental results show that the simple structure adjustment of ResNet-101 network can achieve good segmentation performance on RS images (especially small targets) after the combination of twice augmented RGB channels and nDSM channels respectively. In addition, with the addition of nDSM, the accuracy of buildings and trees with height information has been improved. |
| first_indexed | 2024-12-12T03:18:23Z |
| format | Article |
| id | doaj.art-ab27f784fda044b49402ddd40bd8d74f |
| institution | Directory Open Access Journal |
| issn | 2194-9042 2194-9050 |
| language | English |
| last_indexed | 2024-12-12T03:18:23Z |
| publishDate | 2022-05-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-ab27f784fda044b49402ddd40bd8d74f2022-12-22T00:40:15ZengCopernicus PublicationsISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences2194-90422194-90502022-05-01V-3-2022778310.5194/isprs-annals-V-3-2022-77-2022IMPROVING SEMANTIC SEGMENTATION PERFORMANCE BY JOINTLY USING HIGH RESOLUTION REMOTE SENSING IMAGE AND NDSMR. Yang0Q. Dai1H. Cheng2Y. Zhang3N. Chen4L. Wang5Southwest Forestry University, Kunming, Yunnan province, ChinaSouthwest Forestry University, Kunming, Yunnan province, ChinaSouthwest Forestry University, Kunming, Yunnan province, ChinaSouthwest Forestry University, Kunming, Yunnan province, ChinaSouthwest Forestry University, Kunming, Yunnan province, ChinaSouthwest Forestry University, Kunming, Yunnan province, ChinaSemantic segmentation algorithms based on full convolutional neural network have greatly improved segmentation accuracy of high-resolution remote sensing (RS) images. However, the interpretation of RS images from single sensor is still challenging due to the variety and complexity of land objects, the extreme imbalance distributions of land objects on size and numbers. In contrast, multiple sensors can provide complementary information on the land classes, and thus benefit the interpretation. In this context, this research explores the joint use of RGB optical bands and normalized DSM (nDSM) to analyze an urban scene. The method firstly concatenated three channels RGB image and one channel nDSM band into a four-channel image. Thereafter, ResNet-101 network with fine adjustment were utilized as the backbone network to retain multiple feature information by residual blocks. Then the augmented RGB and nDSM images were used to training the network. The established model was evaluated on the Postdam test set. Results show that the proposed method achieves 86.85% on Overall Accuracy (OA), 77.42% Mean Intersection Over Union (MIOU), which is 6.88% and 11.39% higher than the result achieved by single RGB images. Especially, small targets, such as car and tree, are higher. The experimental results show that the simple structure adjustment of ResNet-101 network can achieve good segmentation performance on RS images (especially small targets) after the combination of twice augmented RGB channels and nDSM channels respectively. In addition, with the addition of nDSM, the accuracy of buildings and trees with height information has been improved.https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-3-2022/77/2022/isprs-annals-V-3-2022-77-2022.pdf |
| spellingShingle | R. Yang Q. Dai H. Cheng Y. Zhang N. Chen L. Wang IMPROVING SEMANTIC SEGMENTATION PERFORMANCE BY JOINTLY USING HIGH RESOLUTION REMOTE SENSING IMAGE AND NDSM ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | IMPROVING SEMANTIC SEGMENTATION PERFORMANCE BY JOINTLY USING HIGH RESOLUTION REMOTE SENSING IMAGE AND NDSM |
| title_full | IMPROVING SEMANTIC SEGMENTATION PERFORMANCE BY JOINTLY USING HIGH RESOLUTION REMOTE SENSING IMAGE AND NDSM |
| title_fullStr | IMPROVING SEMANTIC SEGMENTATION PERFORMANCE BY JOINTLY USING HIGH RESOLUTION REMOTE SENSING IMAGE AND NDSM |
| title_full_unstemmed | IMPROVING SEMANTIC SEGMENTATION PERFORMANCE BY JOINTLY USING HIGH RESOLUTION REMOTE SENSING IMAGE AND NDSM |
| title_short | IMPROVING SEMANTIC SEGMENTATION PERFORMANCE BY JOINTLY USING HIGH RESOLUTION REMOTE SENSING IMAGE AND NDSM |
| title_sort | improving semantic segmentation performance by jointly using high resolution remote sensing image and ndsm |
| url | https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-3-2022/77/2022/isprs-annals-V-3-2022-77-2022.pdf |
| work_keys_str_mv | AT ryang improvingsemanticsegmentationperformancebyjointlyusinghighresolutionremotesensingimageandndsm AT qdai improvingsemanticsegmentationperformancebyjointlyusinghighresolutionremotesensingimageandndsm AT hcheng improvingsemanticsegmentationperformancebyjointlyusinghighresolutionremotesensingimageandndsm AT yzhang improvingsemanticsegmentationperformancebyjointlyusinghighresolutionremotesensingimageandndsm AT nchen improvingsemanticsegmentationperformancebyjointlyusinghighresolutionremotesensingimageandndsm AT lwang improvingsemanticsegmentationperformancebyjointlyusinghighresolutionremotesensingimageandndsm |