Hyperspectral image-aided LiDAR point cloud labeling via spatio-spectral feature representation learning
Urban scene-level 3D point cloud labeling is a very laborious and expensive task compared to images. Conversely however, image processing techniques, deep learning or otherwise are more established and mature. Thus, in a multi-source data environment, the labeling of a point cloud scene via an autom...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-06-01
|
| Series: | International Journal of Applied Earth Observations and Geoinformation |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1569843223001243 |
| _version_ | 1797816076217614336 |
|---|---|
| author | Perpetual Hope Akwensi Zhizhong Kang Ruisheng Wang |
| author_facet | Perpetual Hope Akwensi Zhizhong Kang Ruisheng Wang |
| author_sort | Perpetual Hope Akwensi |
| collection | DOAJ |
| description | Urban scene-level 3D point cloud labeling is a very laborious and expensive task compared to images. Conversely however, image processing techniques, deep learning or otherwise are more established and mature. Thus, in a multi-source data environment, the labeling of a point cloud scene via an automated image process as an initial step, followed by a manual human verification process is an effective way to save man hours and cost. With the above as the goal, this study presents a simple but robust spatio-spectral feature representation approach. In this approach, a class-aware band selection and reduction (CBSR) technique is developed for optimal hyperspectral feature representation. A double-branched convolutional Gaussian Bernoulli deep belief network (CGBDBN) is then used for hierarchical spatial feature extraction from LiDAR-derived data and the CBSR data. Using stacked ensemble learning, spatio-spectral features are generated from the two feature streams via a fusion rule and then classified — the results of which are used in labeling a raw 3D LiDAR point cloud through projection. To evaluate this study, extensive experiments were conducted on the IEEE 2018 Houston dataset — the only publicly available dataset with both hyperspectral image (HSI) and 3D point cloud covering the same area — for urban scene classification. The results indicate that the developed CBSR attained comparatively competitive results with state-of-the-art approaches, thus making it a robust spectral feature representation technique. Also, the weight-sharing property, probabilistic modeling, and hierarchical nature of CGBDBN gives our approach the ability to capture high-level contextual features. Furthermore, compared to the spatial- or spectral-only features, the generated spatio-spectral features are more discriminative and significantly aided in improving the proposed model’s efficacy. Overall, the proposed approach, based on the evaluation metrics, is a robust and effective approach for both coarse- and fine-grained raw LiDAR point cloud labeling tasks. |
| first_indexed | 2024-03-13T08:32:19Z |
| format | Article |
| id | doaj.art-caf0f7e84dd344ccae7da829c8b08d0f |
| institution | Directory Open Access Journal |
| issn | 1569-8432 |
| language | English |
| last_indexed | 2024-03-13T08:32:19Z |
| publishDate | 2023-06-01 |
| publisher | Elsevier |
| record_format | Article |
| series | International Journal of Applied Earth Observations and Geoinformation |
| spelling | doaj.art-caf0f7e84dd344ccae7da829c8b08d0f2023-05-31T04:43:54ZengElsevierInternational Journal of Applied Earth Observations and Geoinformation1569-84322023-06-01120103302Hyperspectral image-aided LiDAR point cloud labeling via spatio-spectral feature representation learningPerpetual Hope Akwensi0Zhizhong Kang1Ruisheng Wang2University of Calgary, 2500 University Drive NW, Calgary T2N 1N4, Alberta, CanadaSchool of Land Science and Technology, China University of Geoscience, Haidian District, Beijing 100083, China; Subcenter of International Cooperation and Research on Lunar and Planetary Exploration, Center of Space Exploration, Ministry of Education of The People’s Republic of China, Beijing 100083, China; Shanxi Key Laboratory of Resources, Environment and Disaster Monitoring, Jinzhong 030600, China; Corresponding author at: School of Land Science and Technology, China University of Geoscience, No. 29 Xueyuan Road, Haidian District, Beijing 100083, China.University of Calgary, 2500 University Drive NW, Calgary T2N 1N4, Alberta, CanadaUrban scene-level 3D point cloud labeling is a very laborious and expensive task compared to images. Conversely however, image processing techniques, deep learning or otherwise are more established and mature. Thus, in a multi-source data environment, the labeling of a point cloud scene via an automated image process as an initial step, followed by a manual human verification process is an effective way to save man hours and cost. With the above as the goal, this study presents a simple but robust spatio-spectral feature representation approach. In this approach, a class-aware band selection and reduction (CBSR) technique is developed for optimal hyperspectral feature representation. A double-branched convolutional Gaussian Bernoulli deep belief network (CGBDBN) is then used for hierarchical spatial feature extraction from LiDAR-derived data and the CBSR data. Using stacked ensemble learning, spatio-spectral features are generated from the two feature streams via a fusion rule and then classified — the results of which are used in labeling a raw 3D LiDAR point cloud through projection. To evaluate this study, extensive experiments were conducted on the IEEE 2018 Houston dataset — the only publicly available dataset with both hyperspectral image (HSI) and 3D point cloud covering the same area — for urban scene classification. The results indicate that the developed CBSR attained comparatively competitive results with state-of-the-art approaches, thus making it a robust spectral feature representation technique. Also, the weight-sharing property, probabilistic modeling, and hierarchical nature of CGBDBN gives our approach the ability to capture high-level contextual features. Furthermore, compared to the spatial- or spectral-only features, the generated spatio-spectral features are more discriminative and significantly aided in improving the proposed model’s efficacy. Overall, the proposed approach, based on the evaluation metrics, is a robust and effective approach for both coarse- and fine-grained raw LiDAR point cloud labeling tasks.http://www.sciencedirect.com/science/article/pii/S1569843223001243Airborne LiDARHyperspectral imageDimensionality reductionConvolutional deep belief networkStack ensemble learning |
| spellingShingle | Perpetual Hope Akwensi Zhizhong Kang Ruisheng Wang Hyperspectral image-aided LiDAR point cloud labeling via spatio-spectral feature representation learning International Journal of Applied Earth Observations and Geoinformation Airborne LiDAR Hyperspectral image Dimensionality reduction Convolutional deep belief network Stack ensemble learning |
| title | Hyperspectral image-aided LiDAR point cloud labeling via spatio-spectral feature representation learning |
| title_full | Hyperspectral image-aided LiDAR point cloud labeling via spatio-spectral feature representation learning |
| title_fullStr | Hyperspectral image-aided LiDAR point cloud labeling via spatio-spectral feature representation learning |
| title_full_unstemmed | Hyperspectral image-aided LiDAR point cloud labeling via spatio-spectral feature representation learning |
| title_short | Hyperspectral image-aided LiDAR point cloud labeling via spatio-spectral feature representation learning |
| title_sort | hyperspectral image aided lidar point cloud labeling via spatio spectral feature representation learning |
| topic | Airborne LiDAR Hyperspectral image Dimensionality reduction Convolutional deep belief network Stack ensemble learning |
| url | http://www.sciencedirect.com/science/article/pii/S1569843223001243 |
| work_keys_str_mv | AT perpetualhopeakwensi hyperspectralimageaidedlidarpointcloudlabelingviaspatiospectralfeaturerepresentationlearning AT zhizhongkang hyperspectralimageaidedlidarpointcloudlabelingviaspatiospectralfeaturerepresentationlearning AT ruishengwang hyperspectralimageaidedlidarpointcloudlabelingviaspatiospectralfeaturerepresentationlearning |