RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration
As an important part of a factory’s automated production line, industrial robots can perform a variety of tasks by integrating external sensors. Among these tasks, grasping scattered workpieces on the industrial assembly line has always been a prominent and difficult point in robot manipul...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/19/8/1873 |
| _version_ | 1828352415650283520 |
|---|---|
| author | Hui Xu Guodong Chen Zhenhua Wang Lining Sun Fan Su |
| author_facet | Hui Xu Guodong Chen Zhenhua Wang Lining Sun Fan Su |
| author_sort | Hui Xu |
| collection | DOAJ |
| description | As an important part of a factory’s automated production line, industrial robots can perform a variety of tasks by integrating external sensors. Among these tasks, grasping scattered workpieces on the industrial assembly line has always been a prominent and difficult point in robot manipulation research. By using RGB-D (color and depth) information, we propose an efficient and practical solution that fuses the approaches of semantic segmentation and point cloud registration to perform object recognition and pose estimation. Different from objects in an indoor environment, the characteristics of the workpiece are relatively simple; thus, we create and label an RGB image dataset from a variety of industrial scenarios and train the modified FCN (Fully Convolutional Network) on a homemade dataset to infer the semantic segmentation results of the input images. Then, we determine the point cloud of the workpieces by incorporating the depth information to estimate the real-time pose of the workpieces. To evaluate the accuracy of the solution, we propose a novel pose error evaluation method based on the robot vision system. This method does not rely on expensive measuring equipment and can also obtain accurate evaluation results. In an industrial scenario, our solution has a rotation error less than two degrees and a translation error < 10 mm. |
| first_indexed | 2024-04-14T01:53:12Z |
| format | Article |
| id | doaj.art-1c3752b04e474f5587e2f5c9cd24f06c |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-14T01:53:12Z |
| publishDate | 2019-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-1c3752b04e474f5587e2f5c9cd24f06c2022-12-22T02:19:14ZengMDPI AGSensors1424-82202019-04-01198187310.3390/s19081873s19081873RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud RegistrationHui Xu0Guodong Chen1Zhenhua Wang2Lining Sun3Fan Su4School of Mechanical and Electric Engineering, Jiangsu Provincial Key Laboratory of Advanced Robotics, Collaborative Innovation Center of Suzhou Nano Science and Technology, Soochow University, Suzhou 215123, ChinaSchool of Mechanical and Electric Engineering, Jiangsu Provincial Key Laboratory of Advanced Robotics, Collaborative Innovation Center of Suzhou Nano Science and Technology, Soochow University, Suzhou 215123, ChinaSchool of Mechanical and Electric Engineering, Jiangsu Provincial Key Laboratory of Advanced Robotics, Collaborative Innovation Center of Suzhou Nano Science and Technology, Soochow University, Suzhou 215123, ChinaSchool of Mechanical and Electric Engineering, Jiangsu Provincial Key Laboratory of Advanced Robotics, Collaborative Innovation Center of Suzhou Nano Science and Technology, Soochow University, Suzhou 215123, ChinaSchool of Mechanical and Electric Engineering, Jiangsu Provincial Key Laboratory of Advanced Robotics, Collaborative Innovation Center of Suzhou Nano Science and Technology, Soochow University, Suzhou 215123, ChinaAs an important part of a factory’s automated production line, industrial robots can perform a variety of tasks by integrating external sensors. Among these tasks, grasping scattered workpieces on the industrial assembly line has always been a prominent and difficult point in robot manipulation research. By using RGB-D (color and depth) information, we propose an efficient and practical solution that fuses the approaches of semantic segmentation and point cloud registration to perform object recognition and pose estimation. Different from objects in an indoor environment, the characteristics of the workpiece are relatively simple; thus, we create and label an RGB image dataset from a variety of industrial scenarios and train the modified FCN (Fully Convolutional Network) on a homemade dataset to infer the semantic segmentation results of the input images. Then, we determine the point cloud of the workpieces by incorporating the depth information to estimate the real-time pose of the workpieces. To evaluate the accuracy of the solution, we propose a novel pose error evaluation method based on the robot vision system. This method does not rely on expensive measuring equipment and can also obtain accurate evaluation results. In an industrial scenario, our solution has a rotation error less than two degrees and a translation error < 10 mm.https://www.mdpi.com/1424-8220/19/8/1873RGB-Dindustrial scenariospose estimationsemantic segmentationhomemade datasetpoint cloud registrationrobot vision system |
| spellingShingle | Hui Xu Guodong Chen Zhenhua Wang Lining Sun Fan Su RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration Sensors RGB-D industrial scenarios pose estimation semantic segmentation homemade dataset point cloud registration robot vision system |
| title | RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration |
| title_full | RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration |
| title_fullStr | RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration |
| title_full_unstemmed | RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration |
| title_short | RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration |
| title_sort | rgb d based pose estimation of workpieces with semantic segmentation and point cloud registration |
| topic | RGB-D industrial scenarios pose estimation semantic segmentation homemade dataset point cloud registration robot vision system |
| url | https://www.mdpi.com/1424-8220/19/8/1873 |
| work_keys_str_mv | AT huixu rgbdbasedposeestimationofworkpieceswithsemanticsegmentationandpointcloudregistration AT guodongchen rgbdbasedposeestimationofworkpieceswithsemanticsegmentationandpointcloudregistration AT zhenhuawang rgbdbasedposeestimationofworkpieceswithsemanticsegmentationandpointcloudregistration AT liningsun rgbdbasedposeestimationofworkpieceswithsemanticsegmentationandpointcloudregistration AT fansu rgbdbasedposeestimationofworkpieceswithsemanticsegmentationandpointcloudregistration |