Mobile robot 3D trajectory estimation on a multilevel surface with multimodal fusion of 2D camera features and a 3D light detection and ranging point cloud
Nowadays, multi-sensor fusion is a popular tool for feature recognition and object detection. Integrating various sensors allows us to obtain reliable information about the environment. This article proposes a 3D robot trajectory estimation based on a multimodal fusion of 2D features extracted from...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2022-03-01
|
| Series: | International Journal of Advanced Robotic Systems |
| Online Access: | https://doi.org/10.1177/17298806221089198 |
| _version_ | 1818775630195458048 |
|---|---|
| author | Vinicio Rosas-Cervantes Quoc-Dong Hoang Sooho Woo Soon-Geul Lee |
| author_facet | Vinicio Rosas-Cervantes Quoc-Dong Hoang Sooho Woo Soon-Geul Lee |
| author_sort | Vinicio Rosas-Cervantes |
| collection | DOAJ |
| description | Nowadays, multi-sensor fusion is a popular tool for feature recognition and object detection. Integrating various sensors allows us to obtain reliable information about the environment. This article proposes a 3D robot trajectory estimation based on a multimodal fusion of 2D features extracted from color images and 3D features from 3D point clouds. First, a set of images was collected using a monocular camera, and we trained a Faster Region Convolutional Neural Network. Using the Faster Region Convolutional Neural Network, the robot detects 2D features from camera input and 3D features using the point’s normal distribution on the 3D point cloud. Then, by matching 2D image features to a 3D point cloud, the robot estimates its position. To validate our results, we compared the trained neural network with similar convolutional neural networks. Then, we evaluated their response for the mobile robot trajectory estimation. |
| first_indexed | 2024-12-18T11:00:05Z |
| format | Article |
| id | doaj.art-4b592face3544de3a801df300aba56d3 |
| institution | Directory Open Access Journal |
| issn | 1729-8814 |
| language | English |
| last_indexed | 2024-12-18T11:00:05Z |
| publishDate | 2022-03-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | International Journal of Advanced Robotic Systems |
| spelling | doaj.art-4b592face3544de3a801df300aba56d32022-12-21T21:10:14ZengSAGE PublishingInternational Journal of Advanced Robotic Systems1729-88142022-03-011910.1177/17298806221089198Mobile robot 3D trajectory estimation on a multilevel surface with multimodal fusion of 2D camera features and a 3D light detection and ranging point cloudVinicio Rosas-Cervantes0Quoc-Dong Hoang1Sooho Woo2Soon-Geul Lee3 Department of Mechanical Engineering, Kyung Hee University, Yongin, South Korea Department of Mechanical Engineering, Kyung Hee University, Yongin, South Korea Department of Mechanical Engineering, Kyung Hee University, Yongin, South Korea Department of Mechanical Engineering, Kyung Hee University, Yongin, South KoreaNowadays, multi-sensor fusion is a popular tool for feature recognition and object detection. Integrating various sensors allows us to obtain reliable information about the environment. This article proposes a 3D robot trajectory estimation based on a multimodal fusion of 2D features extracted from color images and 3D features from 3D point clouds. First, a set of images was collected using a monocular camera, and we trained a Faster Region Convolutional Neural Network. Using the Faster Region Convolutional Neural Network, the robot detects 2D features from camera input and 3D features using the point’s normal distribution on the 3D point cloud. Then, by matching 2D image features to a 3D point cloud, the robot estimates its position. To validate our results, we compared the trained neural network with similar convolutional neural networks. Then, we evaluated their response for the mobile robot trajectory estimation.https://doi.org/10.1177/17298806221089198 |
| spellingShingle | Vinicio Rosas-Cervantes Quoc-Dong Hoang Sooho Woo Soon-Geul Lee Mobile robot 3D trajectory estimation on a multilevel surface with multimodal fusion of 2D camera features and a 3D light detection and ranging point cloud International Journal of Advanced Robotic Systems |
| title | Mobile robot 3D trajectory estimation on a multilevel surface with multimodal fusion of 2D camera features and a 3D light detection and ranging point cloud |
| title_full | Mobile robot 3D trajectory estimation on a multilevel surface with multimodal fusion of 2D camera features and a 3D light detection and ranging point cloud |
| title_fullStr | Mobile robot 3D trajectory estimation on a multilevel surface with multimodal fusion of 2D camera features and a 3D light detection and ranging point cloud |
| title_full_unstemmed | Mobile robot 3D trajectory estimation on a multilevel surface with multimodal fusion of 2D camera features and a 3D light detection and ranging point cloud |
| title_short | Mobile robot 3D trajectory estimation on a multilevel surface with multimodal fusion of 2D camera features and a 3D light detection and ranging point cloud |
| title_sort | mobile robot 3d trajectory estimation on a multilevel surface with multimodal fusion of 2d camera features and a 3d light detection and ranging point cloud |
| url | https://doi.org/10.1177/17298806221089198 |
| work_keys_str_mv | AT viniciorosascervantes mobilerobot3dtrajectoryestimationonamultilevelsurfacewithmultimodalfusionof2dcamerafeaturesanda3dlightdetectionandrangingpointcloud AT quocdonghoang mobilerobot3dtrajectoryestimationonamultilevelsurfacewithmultimodalfusionof2dcamerafeaturesanda3dlightdetectionandrangingpointcloud AT soohowoo mobilerobot3dtrajectoryestimationonamultilevelsurfacewithmultimodalfusionof2dcamerafeaturesanda3dlightdetectionandrangingpointcloud AT soongeullee mobilerobot3dtrajectoryestimationonamultilevelsurfacewithmultimodalfusionof2dcamerafeaturesanda3dlightdetectionandrangingpointcloud |