Lidar-Based 3D Obstacle Detection Using Focal Voxel R-CNN for Farmland Environment
With advances in precision agriculture, autonomous agricultural machines can reduce human labor, optimize workflow, and increase productivity. Accurate and reliable obstacle-detection and avoidance systems are essential for ensuring the safety of automated agricultural machines. Existing LiDAR-based...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Agronomy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-4395/13/3/650 |
| _version_ | 1797614085452333056 |
|---|---|
| author | Jia Qin Ruizhi Sun Kun Zhou Yuanyuan Xu Banghao Lin Lili Yang Zhibo Chen Long Wen Caicong Wu |
| author_facet | Jia Qin Ruizhi Sun Kun Zhou Yuanyuan Xu Banghao Lin Lili Yang Zhibo Chen Long Wen Caicong Wu |
| author_sort | Jia Qin |
| collection | DOAJ |
| description | With advances in precision agriculture, autonomous agricultural machines can reduce human labor, optimize workflow, and increase productivity. Accurate and reliable obstacle-detection and avoidance systems are essential for ensuring the safety of automated agricultural machines. Existing LiDAR-based obstacle detection methods for the farmland environment process the point clouds via manually designed features, which is time-consuming, labor-intensive, and weak in terms of generalization. In contrast, deep learning has a powerful ability to learn features autonomously. In this study, we attempted to apply deep learning in LiDAR-based 3D obstacle detection for the farmland environment. In terms of perception hardware, we established a data acquisition platform including LiDAR, a camera, and a GNSS/INS on the agricultural machine. In terms of perception method, considering the different agricultural conditions, we used our datasets to train an effective 3D obstacle detector, known as Focal Voxel R-CNN. We used focal sparse convolution to replace the original 3D sparse convolution because of its adaptable ability to extract effective features from sparse point cloud data. Specifically, a branch of submanifold sparse convolution was added to the upstream of the backbone convolution network; this adds weight to the foreground point and retains more valuable information. In comparison with Voxel R-CNN, the proposed Focal Voxel R-CNN significantly improves the detection performance for small objects, and the AP in the pedestrian class increased from 89.04% to 92.89%. The results show that our model obtains an mAP of 91.43%, which is 3.36% higher than the base model. The detection speed is 28.57 FPS, which is 4.18 FPS faster than the base model. The experiments show the effectiveness of our model, which can provide a more reliable obstacle detection model for autonomous agricultural machines. |
| first_indexed | 2024-03-11T07:03:40Z |
| format | Article |
| id | doaj.art-99bcb480b17b472184737a6b2b5a1633 |
| institution | Directory Open Access Journal |
| issn | 2073-4395 |
| language | English |
| last_indexed | 2024-03-11T07:03:40Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Agronomy |
| spelling | doaj.art-99bcb480b17b472184737a6b2b5a16332023-11-17T09:04:10ZengMDPI AGAgronomy2073-43952023-02-0113365010.3390/agronomy13030650Lidar-Based 3D Obstacle Detection Using Focal Voxel R-CNN for Farmland EnvironmentJia Qin0Ruizhi Sun1Kun Zhou2Yuanyuan Xu3Banghao Lin4Lili Yang5Zhibo Chen6Long Wen7Caicong Wu8College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, ChinaCollege of Information and Electrical Engineering, China Agricultural University, Beijing 100083, ChinaResearch & Advanced Engineering, AGCO A/S, DK-8930 Randers, DenmarkCollege of Information and Electrical Engineering, China Agricultural University, Beijing 100083, ChinaCollege of Information and Electrical Engineering, China Agricultural University, Beijing 100083, ChinaCollege of Information and Electrical Engineering, China Agricultural University, Beijing 100083, ChinaCollege of Information and Electrical Engineering, China Agricultural University, Beijing 100083, ChinaCollege of Information and Electrical Engineering, China Agricultural University, Beijing 100083, ChinaCollege of Information and Electrical Engineering, China Agricultural University, Beijing 100083, ChinaWith advances in precision agriculture, autonomous agricultural machines can reduce human labor, optimize workflow, and increase productivity. Accurate and reliable obstacle-detection and avoidance systems are essential for ensuring the safety of automated agricultural machines. Existing LiDAR-based obstacle detection methods for the farmland environment process the point clouds via manually designed features, which is time-consuming, labor-intensive, and weak in terms of generalization. In contrast, deep learning has a powerful ability to learn features autonomously. In this study, we attempted to apply deep learning in LiDAR-based 3D obstacle detection for the farmland environment. In terms of perception hardware, we established a data acquisition platform including LiDAR, a camera, and a GNSS/INS on the agricultural machine. In terms of perception method, considering the different agricultural conditions, we used our datasets to train an effective 3D obstacle detector, known as Focal Voxel R-CNN. We used focal sparse convolution to replace the original 3D sparse convolution because of its adaptable ability to extract effective features from sparse point cloud data. Specifically, a branch of submanifold sparse convolution was added to the upstream of the backbone convolution network; this adds weight to the foreground point and retains more valuable information. In comparison with Voxel R-CNN, the proposed Focal Voxel R-CNN significantly improves the detection performance for small objects, and the AP in the pedestrian class increased from 89.04% to 92.89%. The results show that our model obtains an mAP of 91.43%, which is 3.36% higher than the base model. The detection speed is 28.57 FPS, which is 4.18 FPS faster than the base model. The experiments show the effectiveness of our model, which can provide a more reliable obstacle detection model for autonomous agricultural machines.https://www.mdpi.com/2073-4395/13/3/650obstacle detectionLiDARpoint cloudsfocal voxel R-CNNfarmland |
| spellingShingle | Jia Qin Ruizhi Sun Kun Zhou Yuanyuan Xu Banghao Lin Lili Yang Zhibo Chen Long Wen Caicong Wu Lidar-Based 3D Obstacle Detection Using Focal Voxel R-CNN for Farmland Environment Agronomy obstacle detection LiDAR point clouds focal voxel R-CNN farmland |
| title | Lidar-Based 3D Obstacle Detection Using Focal Voxel R-CNN for Farmland Environment |
| title_full | Lidar-Based 3D Obstacle Detection Using Focal Voxel R-CNN for Farmland Environment |
| title_fullStr | Lidar-Based 3D Obstacle Detection Using Focal Voxel R-CNN for Farmland Environment |
| title_full_unstemmed | Lidar-Based 3D Obstacle Detection Using Focal Voxel R-CNN for Farmland Environment |
| title_short | Lidar-Based 3D Obstacle Detection Using Focal Voxel R-CNN for Farmland Environment |
| title_sort | lidar based 3d obstacle detection using focal voxel r cnn for farmland environment |
| topic | obstacle detection LiDAR point clouds focal voxel R-CNN farmland |
| url | https://www.mdpi.com/2073-4395/13/3/650 |
| work_keys_str_mv | AT jiaqin lidarbased3dobstacledetectionusingfocalvoxelrcnnforfarmlandenvironment AT ruizhisun lidarbased3dobstacledetectionusingfocalvoxelrcnnforfarmlandenvironment AT kunzhou lidarbased3dobstacledetectionusingfocalvoxelrcnnforfarmlandenvironment AT yuanyuanxu lidarbased3dobstacledetectionusingfocalvoxelrcnnforfarmlandenvironment AT banghaolin lidarbased3dobstacledetectionusingfocalvoxelrcnnforfarmlandenvironment AT liliyang lidarbased3dobstacledetectionusingfocalvoxelrcnnforfarmlandenvironment AT zhibochen lidarbased3dobstacledetectionusingfocalvoxelrcnnforfarmlandenvironment AT longwen lidarbased3dobstacledetectionusingfocalvoxelrcnnforfarmlandenvironment AT caicongwu lidarbased3dobstacledetectionusingfocalvoxelrcnnforfarmlandenvironment |