Deep-Learning-Based Three-Dimensional Detection of Individual Wave-Dissipating Blocks from As-Built Point Clouds Measured by UAV Photogrammetry and Multibeam Echo-Sounder
Wave-dissipating blocks are the armor elements of breakwaters that protect beaches, ports, and harbors from erosion by waves. Monitoring the poses of individual wave-dissipating blocks benefits the accuracy of the block supplemental work plan, recording of the construction status, and monitoring of...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/21/5575 |
| _version_ | 1797466609285070848 |
|---|---|
| author | Yajun Xu Satoshi Kanai Hiroaki Date Tomoaki Sano |
| author_facet | Yajun Xu Satoshi Kanai Hiroaki Date Tomoaki Sano |
| author_sort | Yajun Xu |
| collection | DOAJ |
| description | Wave-dissipating blocks are the armor elements of breakwaters that protect beaches, ports, and harbors from erosion by waves. Monitoring the poses of individual wave-dissipating blocks benefits the accuracy of the block supplemental work plan, recording of the construction status, and monitoring of long-term pose change in blocks. This study proposes a deep-learning-based approach to detect individual blocks from large-scale three-dimensional point clouds measured with a pile of wave-dissipating blocks placed overseas and underseas using UAV photogrammetry and a multibeam echo-sounder. The approach comprises three main steps. First, the instance segmentation using our originally designed deep convolutional neural network partitions an original point cloud into small subsets of points, each corresponding to an individual block. Then, the block-wise 6D pose is estimated using a three-dimensional feature descriptor, point cloud registration, and CAD models of blocks. Finally, the type of each segmented block is identified using model registration results. The results of the instance segmentation on real-world and synthetic point cloud data achieved 70–90% precision and 50–76% recall with an intersection of union threshold of 0.5. The pose estimation results on synthetic data achieved 83–95% precision and 77–95% recall under strict pose criteria. The average block-wise displacement error was 30 mm, and the rotation error was less than <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mn>2</mn><mo>∘</mo></msup></semantics></math></inline-formula>. The pose estimation results on real-world data showed that the fitting error between the reconstructed scene and the scene point cloud ranged between 30 and 50 mm, which is below 2% of the detected block size. The accuracy in the block-type classification on real-world point clouds reached about 95%. These block detection performances demonstrate the effectiveness of our approach. |
| first_indexed | 2024-03-09T18:41:11Z |
| format | Article |
| id | doaj.art-8dbf1fe58be34f57ad9c364793d8ef44 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T18:41:11Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-8dbf1fe58be34f57ad9c364793d8ef442023-11-24T06:41:16ZengMDPI AGRemote Sensing2072-42922022-11-011421557510.3390/rs14215575Deep-Learning-Based Three-Dimensional Detection of Individual Wave-Dissipating Blocks from As-Built Point Clouds Measured by UAV Photogrammetry and Multibeam Echo-SounderYajun Xu0Satoshi Kanai1Hiroaki Date2Tomoaki Sano3Graduate School of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita Ward, Sapporo 060-0814, Hokkaido, JapanGraduate School of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita Ward, Sapporo 060-0814, Hokkaido, JapanGraduate School of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita Ward, Sapporo 060-0814, Hokkaido, JapanAlpha Hydraulic Engineering Consultants Co., Ltd., 14 Chome-516-336 Hassamu 9 Jo, Nishi Ward, Sapporo 063-0829, Hokkaido, JapanWave-dissipating blocks are the armor elements of breakwaters that protect beaches, ports, and harbors from erosion by waves. Monitoring the poses of individual wave-dissipating blocks benefits the accuracy of the block supplemental work plan, recording of the construction status, and monitoring of long-term pose change in blocks. This study proposes a deep-learning-based approach to detect individual blocks from large-scale three-dimensional point clouds measured with a pile of wave-dissipating blocks placed overseas and underseas using UAV photogrammetry and a multibeam echo-sounder. The approach comprises three main steps. First, the instance segmentation using our originally designed deep convolutional neural network partitions an original point cloud into small subsets of points, each corresponding to an individual block. Then, the block-wise 6D pose is estimated using a three-dimensional feature descriptor, point cloud registration, and CAD models of blocks. Finally, the type of each segmented block is identified using model registration results. The results of the instance segmentation on real-world and synthetic point cloud data achieved 70–90% precision and 50–76% recall with an intersection of union threshold of 0.5. The pose estimation results on synthetic data achieved 83–95% precision and 77–95% recall under strict pose criteria. The average block-wise displacement error was 30 mm, and the rotation error was less than <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mn>2</mn><mo>∘</mo></msup></semantics></math></inline-formula>. The pose estimation results on real-world data showed that the fitting error between the reconstructed scene and the scene point cloud ranged between 30 and 50 mm, which is below 2% of the detected block size. The accuracy in the block-type classification on real-world point clouds reached about 95%. These block detection performances demonstrate the effectiveness of our approach.https://www.mdpi.com/2072-4292/14/21/5575wave-dissipating blockspoint cloudinstance segmentationdeep learningCAD modelpose estimation |
| spellingShingle | Yajun Xu Satoshi Kanai Hiroaki Date Tomoaki Sano Deep-Learning-Based Three-Dimensional Detection of Individual Wave-Dissipating Blocks from As-Built Point Clouds Measured by UAV Photogrammetry and Multibeam Echo-Sounder Remote Sensing wave-dissipating blocks point cloud instance segmentation deep learning CAD model pose estimation |
| title | Deep-Learning-Based Three-Dimensional Detection of Individual Wave-Dissipating Blocks from As-Built Point Clouds Measured by UAV Photogrammetry and Multibeam Echo-Sounder |
| title_full | Deep-Learning-Based Three-Dimensional Detection of Individual Wave-Dissipating Blocks from As-Built Point Clouds Measured by UAV Photogrammetry and Multibeam Echo-Sounder |
| title_fullStr | Deep-Learning-Based Three-Dimensional Detection of Individual Wave-Dissipating Blocks from As-Built Point Clouds Measured by UAV Photogrammetry and Multibeam Echo-Sounder |
| title_full_unstemmed | Deep-Learning-Based Three-Dimensional Detection of Individual Wave-Dissipating Blocks from As-Built Point Clouds Measured by UAV Photogrammetry and Multibeam Echo-Sounder |
| title_short | Deep-Learning-Based Three-Dimensional Detection of Individual Wave-Dissipating Blocks from As-Built Point Clouds Measured by UAV Photogrammetry and Multibeam Echo-Sounder |
| title_sort | deep learning based three dimensional detection of individual wave dissipating blocks from as built point clouds measured by uav photogrammetry and multibeam echo sounder |
| topic | wave-dissipating blocks point cloud instance segmentation deep learning CAD model pose estimation |
| url | https://www.mdpi.com/2072-4292/14/21/5575 |
| work_keys_str_mv | AT yajunxu deeplearningbasedthreedimensionaldetectionofindividualwavedissipatingblocksfromasbuiltpointcloudsmeasuredbyuavphotogrammetryandmultibeamechosounder AT satoshikanai deeplearningbasedthreedimensionaldetectionofindividualwavedissipatingblocksfromasbuiltpointcloudsmeasuredbyuavphotogrammetryandmultibeamechosounder AT hiroakidate deeplearningbasedthreedimensionaldetectionofindividualwavedissipatingblocksfromasbuiltpointcloudsmeasuredbyuavphotogrammetryandmultibeamechosounder AT tomoakisano deeplearningbasedthreedimensionaldetectionofindividualwavedissipatingblocksfromasbuiltpointcloudsmeasuredbyuavphotogrammetryandmultibeamechosounder |