Point cloud completion in challenging indoor scenarios with human motion
Combining and completing point cloud data from two or more sensors with arbitrarily relative perspectives in a dynamic, cluttered, and complex environment is challenging, especially when the two sensors have significant perspective differences while the large overlap ratio and feature-rich scene can...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-05-01
|
| Series: | Frontiers in Robotics and AI |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frobt.2023.1184614/full |
| _version_ | 1827950611205718016 |
|---|---|
| author | Chengsi Zhang Stephen Czarnuch |
| author_facet | Chengsi Zhang Stephen Czarnuch |
| author_sort | Chengsi Zhang |
| collection | DOAJ |
| description | Combining and completing point cloud data from two or more sensors with arbitrarily relative perspectives in a dynamic, cluttered, and complex environment is challenging, especially when the two sensors have significant perspective differences while the large overlap ratio and feature-rich scene cannot be guaranteed. We create a novel approach targeting this challenging scenario by registering two camera captures in a time series with unknown perspectives and human movements to easily use our system in a real-life scene. In our approach, we first reduce the six unknowns of 3D point cloud completion to three by aligning the ground planes found by our previous perspective-independent 3D ground plane estimation algorithm. Subsequently, we use a histogram-based approach to identify and extract all the humans from each frame generating a three-dimensional (3D) human walking sequence in a time series. To enhance accuracy and performance, we convert 3D human walking sequences to lines by calculating the center of mass (CoM) point of each human body and connecting them. Finally, we match the walking paths in different data trials by minimizing the Fréchet distance between two walking paths and using 2D iterative closest point (ICP) to find the remaining three unknowns in the overall transformation matrix for the final alignment. Using this approach, we can successfully register the corresponding walking path of the human between the two cameras’ captures and estimate the transformation matrix between the two sensors. |
| first_indexed | 2024-04-09T13:28:16Z |
| format | Article |
| id | doaj.art-2b6eb8c6b9db4bdeb89f2e9f3bda7b26 |
| institution | Directory Open Access Journal |
| issn | 2296-9144 |
| language | English |
| last_indexed | 2024-04-09T13:28:16Z |
| publishDate | 2023-05-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Robotics and AI |
| spelling | doaj.art-2b6eb8c6b9db4bdeb89f2e9f3bda7b262023-05-10T05:21:33ZengFrontiers Media S.A.Frontiers in Robotics and AI2296-91442023-05-011010.3389/frobt.2023.11846141184614Point cloud completion in challenging indoor scenarios with human motionChengsi Zhang0Stephen Czarnuch1Department of Electrical and Computer Engineering, Faculty of Engineering and Applied Science, Memorial University of Newfoundland, St. John’s, NL, CanadaDepartment of Electrical and Computer Engineering, Faculty of Engineering and Applied Science and the Discipline of Emergency Medicine, Faculty of Medicine, Memorial University of Newfoundland, St. John’s, NL, CanadaCombining and completing point cloud data from two or more sensors with arbitrarily relative perspectives in a dynamic, cluttered, and complex environment is challenging, especially when the two sensors have significant perspective differences while the large overlap ratio and feature-rich scene cannot be guaranteed. We create a novel approach targeting this challenging scenario by registering two camera captures in a time series with unknown perspectives and human movements to easily use our system in a real-life scene. In our approach, we first reduce the six unknowns of 3D point cloud completion to three by aligning the ground planes found by our previous perspective-independent 3D ground plane estimation algorithm. Subsequently, we use a histogram-based approach to identify and extract all the humans from each frame generating a three-dimensional (3D) human walking sequence in a time series. To enhance accuracy and performance, we convert 3D human walking sequences to lines by calculating the center of mass (CoM) point of each human body and connecting them. Finally, we match the walking paths in different data trials by minimizing the Fréchet distance between two walking paths and using 2D iterative closest point (ICP) to find the remaining three unknowns in the overall transformation matrix for the final alignment. Using this approach, we can successfully register the corresponding walking path of the human between the two cameras’ captures and estimate the transformation matrix between the two sensors.https://www.frontiersin.org/articles/10.3389/frobt.2023.1184614/full3D completionpoint cloud registrationpoint cloud segmentation3D data analysis3D data processing |
| spellingShingle | Chengsi Zhang Stephen Czarnuch Point cloud completion in challenging indoor scenarios with human motion Frontiers in Robotics and AI 3D completion point cloud registration point cloud segmentation 3D data analysis 3D data processing |
| title | Point cloud completion in challenging indoor scenarios with human motion |
| title_full | Point cloud completion in challenging indoor scenarios with human motion |
| title_fullStr | Point cloud completion in challenging indoor scenarios with human motion |
| title_full_unstemmed | Point cloud completion in challenging indoor scenarios with human motion |
| title_short | Point cloud completion in challenging indoor scenarios with human motion |
| title_sort | point cloud completion in challenging indoor scenarios with human motion |
| topic | 3D completion point cloud registration point cloud segmentation 3D data analysis 3D data processing |
| url | https://www.frontiersin.org/articles/10.3389/frobt.2023.1184614/full |
| work_keys_str_mv | AT chengsizhang pointcloudcompletioninchallengingindoorscenarioswithhumanmotion AT stephenczarnuch pointcloudcompletioninchallengingindoorscenarioswithhumanmotion |