An Onboard Point Cloud Semantic Segmentation System for Robotic Platforms
Point clouds represent an important way for robots to perceive their environments, and can be acquired by mobile robots with LiDAR sensors or underwater robots with sonar sensors. Hence, real-time semantic segmentation of point clouds with onboard edge devices is essential for robots to apprehend th...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-05-01
|
| Series: | Machines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1702/11/5/571 |
| _version_ | 1797599350285664256 |
|---|---|
| author | Fei Wang Yujie Yang Jingchun Zhou Weishi Zhang |
| author_facet | Fei Wang Yujie Yang Jingchun Zhou Weishi Zhang |
| author_sort | Fei Wang |
| collection | DOAJ |
| description | Point clouds represent an important way for robots to perceive their environments, and can be acquired by mobile robots with LiDAR sensors or underwater robots with sonar sensors. Hence, real-time semantic segmentation of point clouds with onboard edge devices is essential for robots to apprehend their surroundings. In this paper, we propose an onboard point cloud semantic segmentation system for robotic platforms to overcome the conflict between attaining high accuracy of segmentation results and the limited available computational resources of onboard devices. Our system takes raw a sequence of point clouds as inputs, and outputs semantic segmentation results for each frame as well as a reconstructed semantic map of the environment. At the core of our system is the transformer-based hierarchical feature extraction module and fusion module. The two modules are implemented with sparse tensor technologies to speed up inference. The predictions are accumulated according to Bayes rules to generate a global semantic map. Experimental results on the SemanticKITTI dataset show that our system achieves +2.2% mIoU and 18× speed improvements compared with SOTA methods. Our system is able to process 2.2 M points per second on Jetson AGX Xavier (NVIDIA, Santa Clara, USA), demonstrating its applicability to various robotic platforms. |
| first_indexed | 2024-03-11T03:33:06Z |
| format | Article |
| id | doaj.art-19699ba79dd948b39d828a1a3594d256 |
| institution | Directory Open Access Journal |
| issn | 2075-1702 |
| language | English |
| last_indexed | 2024-03-11T03:33:06Z |
| publishDate | 2023-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machines |
| spelling | doaj.art-19699ba79dd948b39d828a1a3594d2562023-11-18T02:12:03ZengMDPI AGMachines2075-17022023-05-0111557110.3390/machines11050571An Onboard Point Cloud Semantic Segmentation System for Robotic PlatformsFei Wang0Yujie Yang1Jingchun Zhou2Weishi Zhang3College of Information Science and Technology, Dalian Maritime University, Dalian 116000, ChinaCollege of Information Science and Technology, Dalian Maritime University, Dalian 116000, ChinaCollege of Information Science and Technology, Dalian Maritime University, Dalian 116000, ChinaCollege of Information Science and Technology, Dalian Maritime University, Dalian 116000, ChinaPoint clouds represent an important way for robots to perceive their environments, and can be acquired by mobile robots with LiDAR sensors or underwater robots with sonar sensors. Hence, real-time semantic segmentation of point clouds with onboard edge devices is essential for robots to apprehend their surroundings. In this paper, we propose an onboard point cloud semantic segmentation system for robotic platforms to overcome the conflict between attaining high accuracy of segmentation results and the limited available computational resources of onboard devices. Our system takes raw a sequence of point clouds as inputs, and outputs semantic segmentation results for each frame as well as a reconstructed semantic map of the environment. At the core of our system is the transformer-based hierarchical feature extraction module and fusion module. The two modules are implemented with sparse tensor technologies to speed up inference. The predictions are accumulated according to Bayes rules to generate a global semantic map. Experimental results on the SemanticKITTI dataset show that our system achieves +2.2% mIoU and 18× speed improvements compared with SOTA methods. Our system is able to process 2.2 M points per second on Jetson AGX Xavier (NVIDIA, Santa Clara, USA), demonstrating its applicability to various robotic platforms.https://www.mdpi.com/2075-1702/11/5/571semantic segmentationpoint cloudtransformersparse tensorrobot |
| spellingShingle | Fei Wang Yujie Yang Jingchun Zhou Weishi Zhang An Onboard Point Cloud Semantic Segmentation System for Robotic Platforms Machines semantic segmentation point cloud transformer sparse tensor robot |
| title | An Onboard Point Cloud Semantic Segmentation System for Robotic Platforms |
| title_full | An Onboard Point Cloud Semantic Segmentation System for Robotic Platforms |
| title_fullStr | An Onboard Point Cloud Semantic Segmentation System for Robotic Platforms |
| title_full_unstemmed | An Onboard Point Cloud Semantic Segmentation System for Robotic Platforms |
| title_short | An Onboard Point Cloud Semantic Segmentation System for Robotic Platforms |
| title_sort | onboard point cloud semantic segmentation system for robotic platforms |
| topic | semantic segmentation point cloud transformer sparse tensor robot |
| url | https://www.mdpi.com/2075-1702/11/5/571 |
| work_keys_str_mv | AT feiwang anonboardpointcloudsemanticsegmentationsystemforroboticplatforms AT yujieyang anonboardpointcloudsemanticsegmentationsystemforroboticplatforms AT jingchunzhou anonboardpointcloudsemanticsegmentationsystemforroboticplatforms AT weishizhang anonboardpointcloudsemanticsegmentationsystemforroboticplatforms AT feiwang onboardpointcloudsemanticsegmentationsystemforroboticplatforms AT yujieyang onboardpointcloudsemanticsegmentationsystemforroboticplatforms AT jingchunzhou onboardpointcloudsemanticsegmentationsystemforroboticplatforms AT weishizhang onboardpointcloudsemanticsegmentationsystemforroboticplatforms |