Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation
State-of-the-art 3D object detection for autonomous driving is achieved by processing lidar sensor data with deep-learning methods. However, the detection quality of the state of the art is still far from enabling safe driving in all conditions. Additional sensor modalities need to be used to increa...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/6/2599 |
| _version_ | 1797541334108602368 |
|---|---|
| author | Felix Nobis Felix Fent Johannes Betz Markus Lienkamp |
| author_facet | Felix Nobis Felix Fent Johannes Betz Markus Lienkamp |
| author_sort | Felix Nobis |
| collection | DOAJ |
| description | State-of-the-art 3D object detection for autonomous driving is achieved by processing lidar sensor data with deep-learning methods. However, the detection quality of the state of the art is still far from enabling safe driving in all conditions. Additional sensor modalities need to be used to increase the confidence and robustness of the overall detection result. Researchers have recently explored radar data as an additional input source for universal 3D object detection. This paper proposes artificial neural network architectures to segment sparse radar point cloud data. Segmentation is an intermediate step towards radar object detection as a complementary concept to lidar object detection. Conceptually, we adapt Kernel Point Convolution (KPConv) layers for radar data. Additionally, we introduce a long short-term memory (LSTM) variant based on KPConv layers to make use of the information content in the time dimension of radar data. This is motivated by classical radar processing, where tracking of features over time is imperative to generate confident object proposals. We benchmark several variants of the network on the public nuScenes data set against a state-of-the-art pointnet-based approach. The performance of the networks is limited by the quality of the publicly available data. The radar data and radar-label quality is of great importance to the training and evaluation of machine learning models. Therefore, the advantages and disadvantages of the available data set, regarding its radar data, are discussed in detail. The need for a radar-focused data set for object detection is expressed. We assume that higher segmentation scores should be achievable with better-quality data for all models compared, and differences between the models should manifest more clearly. To facilitate research with additional radar data, the modular code for this research will be made available to the public. |
| first_indexed | 2024-03-10T13:14:32Z |
| format | Article |
| id | doaj.art-6d5572441c624ff3aa44fc927ec32ca7 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T13:14:32Z |
| publishDate | 2021-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-6d5572441c624ff3aa44fc927ec32ca72023-11-21T10:30:07ZengMDPI AGApplied Sciences2076-34172021-03-01116259910.3390/app11062599Kernel Point Convolution LSTM Networks for Radar Point Cloud SegmentationFelix Nobis0Felix Fent1Johannes Betz2Markus Lienkamp3Institute of Automotive Technology, Technical University of Munich, 85748 Garching, GermanyInstitute of Automotive Technology, Technical University of Munich, 85748 Garching, GermanymLab:Real-Time and Embedded Systems Lab, University of Pennsylvania, Philadelphia, PA 19104-6243, USAInstitute of Automotive Technology, Technical University of Munich, 85748 Garching, GermanyState-of-the-art 3D object detection for autonomous driving is achieved by processing lidar sensor data with deep-learning methods. However, the detection quality of the state of the art is still far from enabling safe driving in all conditions. Additional sensor modalities need to be used to increase the confidence and robustness of the overall detection result. Researchers have recently explored radar data as an additional input source for universal 3D object detection. This paper proposes artificial neural network architectures to segment sparse radar point cloud data. Segmentation is an intermediate step towards radar object detection as a complementary concept to lidar object detection. Conceptually, we adapt Kernel Point Convolution (KPConv) layers for radar data. Additionally, we introduce a long short-term memory (LSTM) variant based on KPConv layers to make use of the information content in the time dimension of radar data. This is motivated by classical radar processing, where tracking of features over time is imperative to generate confident object proposals. We benchmark several variants of the network on the public nuScenes data set against a state-of-the-art pointnet-based approach. The performance of the networks is limited by the quality of the publicly available data. The radar data and radar-label quality is of great importance to the training and evaluation of machine learning models. Therefore, the advantages and disadvantages of the available data set, regarding its radar data, are discussed in detail. The need for a radar-focused data set for object detection is expressed. We assume that higher segmentation scores should be achievable with better-quality data for all models compared, and differences between the models should manifest more clearly. To facilitate research with additional radar data, the modular code for this research will be made available to the public.https://www.mdpi.com/2076-3417/11/6/2599perceptiondeep learningradar segmentationradar point cloudobject detection |
| spellingShingle | Felix Nobis Felix Fent Johannes Betz Markus Lienkamp Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation Applied Sciences perception deep learning radar segmentation radar point cloud object detection |
| title | Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation |
| title_full | Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation |
| title_fullStr | Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation |
| title_full_unstemmed | Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation |
| title_short | Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation |
| title_sort | kernel point convolution lstm networks for radar point cloud segmentation |
| topic | perception deep learning radar segmentation radar point cloud object detection |
| url | https://www.mdpi.com/2076-3417/11/6/2599 |
| work_keys_str_mv | AT felixnobis kernelpointconvolutionlstmnetworksforradarpointcloudsegmentation AT felixfent kernelpointconvolutionlstmnetworksforradarpointcloudsegmentation AT johannesbetz kernelpointconvolutionlstmnetworksforradarpointcloudsegmentation AT markuslienkamp kernelpointconvolutionlstmnetworksforradarpointcloudsegmentation |