PointBoost: LiDAR-Enhanced Semantic Segmentation of Remote Sensing Imagery
Semantic segmentation of imagery is typically reliant on texture information from raster images, which limits its accuracy due to the inherently 2-D nature of the plane. To address the nonnegligible domain gap between different metric spaces, multimodal methods have been introduced that incorporate...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10154131/ |
| _version_ | 1797785099692933120 |
|---|---|
| author | Yongjun Zhang Yameng Wang Yi Wan Wenming Zhou Bin Zhang |
| author_facet | Yongjun Zhang Yameng Wang Yi Wan Wenming Zhou Bin Zhang |
| author_sort | Yongjun Zhang |
| collection | DOAJ |
| description | Semantic segmentation of imagery is typically reliant on texture information from raster images, which limits its accuracy due to the inherently 2-D nature of the plane. To address the nonnegligible domain gap between different metric spaces, multimodal methods have been introduced that incorporate Light Detection and Ranging (LiDAR) derived feature maps. This converts multimodal joint semantic segmentation between 3-D point clouds and 2-D optical imagery into a feature extraction process for the 2.5-D product, which is achieved by concatenating LiDAR-derived feather maps, such as digital surface models, with the optical images. However, the information sources for these methods are still limited to 2-D, and certain properties of point clouds are lost as a result. In this study, we propose PointBoost, an effective sequential segmentation framework that can work directly with cross-modal data of LiDAR point clouds and imagery, which is able to extract richer semantic features from cross-dimensional and cross-modal information. Ablation experiments demonstrate that PointBoost can take full advantage of the 3-D topological structure between points and attribute information of point clouds, which is often discarded by other methods. Experiments on three multimodal datasets, namely N3C-California, ISPRS Vaihingen, and GRSS DFC 2018, show that our method achieves superior performance with good generalization. |
| first_indexed | 2024-03-13T00:50:14Z |
| format | Article |
| id | doaj.art-596c578a073b44058b18732a2214f2d7 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-03-13T00:50:14Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-596c578a073b44058b18732a2214f2d72023-07-07T23:00:02ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352023-01-01165618562810.1109/JSTARS.2023.328691210154131PointBoost: LiDAR-Enhanced Semantic Segmentation of Remote Sensing ImageryYongjun Zhang0https://orcid.org/0000-0001-9845-4251Yameng Wang1https://orcid.org/0009-0005-7405-1374Yi Wan2https://orcid.org/0000-0001-5492-0564Wenming Zhou3Bin Zhang4https://orcid.org/0000-0001-9545-2760School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaChina Railway Design Corporation, Tianjin, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaSemantic segmentation of imagery is typically reliant on texture information from raster images, which limits its accuracy due to the inherently 2-D nature of the plane. To address the nonnegligible domain gap between different metric spaces, multimodal methods have been introduced that incorporate Light Detection and Ranging (LiDAR) derived feature maps. This converts multimodal joint semantic segmentation between 3-D point clouds and 2-D optical imagery into a feature extraction process for the 2.5-D product, which is achieved by concatenating LiDAR-derived feather maps, such as digital surface models, with the optical images. However, the information sources for these methods are still limited to 2-D, and certain properties of point clouds are lost as a result. In this study, we propose PointBoost, an effective sequential segmentation framework that can work directly with cross-modal data of LiDAR point clouds and imagery, which is able to extract richer semantic features from cross-dimensional and cross-modal information. Ablation experiments demonstrate that PointBoost can take full advantage of the 3-D topological structure between points and attribute information of point clouds, which is often discarded by other methods. Experiments on three multimodal datasets, namely N3C-California, ISPRS Vaihingen, and GRSS DFC 2018, show that our method achieves superior performance with good generalization.https://ieeexplore.ieee.org/document/10154131/Light Detection and Ranging (LiDAR)remote sensing imagerysemantic segmentation |
| spellingShingle | Yongjun Zhang Yameng Wang Yi Wan Wenming Zhou Bin Zhang PointBoost: LiDAR-Enhanced Semantic Segmentation of Remote Sensing Imagery IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Light Detection and Ranging (LiDAR) remote sensing imagery semantic segmentation |
| title | PointBoost: LiDAR-Enhanced Semantic Segmentation of Remote Sensing Imagery |
| title_full | PointBoost: LiDAR-Enhanced Semantic Segmentation of Remote Sensing Imagery |
| title_fullStr | PointBoost: LiDAR-Enhanced Semantic Segmentation of Remote Sensing Imagery |
| title_full_unstemmed | PointBoost: LiDAR-Enhanced Semantic Segmentation of Remote Sensing Imagery |
| title_short | PointBoost: LiDAR-Enhanced Semantic Segmentation of Remote Sensing Imagery |
| title_sort | pointboost lidar enhanced semantic segmentation of remote sensing imagery |
| topic | Light Detection and Ranging (LiDAR) remote sensing imagery semantic segmentation |
| url | https://ieeexplore.ieee.org/document/10154131/ |
| work_keys_str_mv | AT yongjunzhang pointboostlidarenhancedsemanticsegmentationofremotesensingimagery AT yamengwang pointboostlidarenhancedsemanticsegmentationofremotesensingimagery AT yiwan pointboostlidarenhancedsemanticsegmentationofremotesensingimagery AT wenmingzhou pointboostlidarenhancedsemanticsegmentationofremotesensingimagery AT binzhang pointboostlidarenhancedsemanticsegmentationofremotesensingimagery |