3D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared Sensing
The automated driving of agricultural machinery is of great significance for the agricultural production efficiency, yet is still challenging due to the significantly varied environmental conditions through day and night. To address operation safety for pedestrians in farmland, this paper proposes a...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/15/2896 |
| _version_ | 1797525161312780288 |
|---|---|
| author | Wei Tian Zhenwen Deng Dong Yin Zehan Zheng Yuyao Huang Xin Bi |
| author_facet | Wei Tian Zhenwen Deng Dong Yin Zehan Zheng Yuyao Huang Xin Bi |
| author_sort | Wei Tian |
| collection | DOAJ |
| description | The automated driving of agricultural machinery is of great significance for the agricultural production efficiency, yet is still challenging due to the significantly varied environmental conditions through day and night. To address operation safety for pedestrians in farmland, this paper proposes a 3D person sensing approach based on monocular RGB and Far-Infrared (FIR) images. Since public available datasets for agricultural 3D pedestrian detection are scarce, a new dataset is proposed, named as “FieldSafePedestrian”, which includes field images in both day and night. The implemented data augmentations of night images and semi-automatic labeling approach are also elaborated to facilitate the 3D annotation of pedestrians. To fuse heterogeneous images of sensors with non-parallel optical axis, the Dual-Input Depth-Guided Dynamic-Depthwise-Dilated Fusion network (D5F) is proposed, which assists the pixel alignment between FIR and RGB images with estimated depth information and deploys a dynamic filtering to guide the heterogeneous information fusion. Experiments on field images in both daytime and nighttime demonstrate that compared with the state-of-the-arts, the dynamic aligned image fusion achieves an accuracy gain of 3.9% and 4.5% in terms of center distance and BEV-IOU, respectively, without affecting the run-time efficiency. |
| first_indexed | 2024-03-10T09:09:50Z |
| format | Article |
| id | doaj.art-180fec925e944d9f89b3b9eff692fdfa |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T09:09:50Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-180fec925e944d9f89b3b9eff692fdfa2023-11-22T06:05:45ZengMDPI AGRemote Sensing2072-42922021-07-011315289610.3390/rs131528963D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared SensingWei Tian0Zhenwen Deng1Dong Yin2Zehan Zheng3Yuyao Huang4Xin Bi5Institute of Intelligent Vehicles, School of Automotive Studies, Tongji University, Shanghai 201804, ChinaInstitute of Intelligent Vehicles, School of Automotive Studies, Tongji University, Shanghai 201804, ChinaInstitute of Intelligent Vehicles, School of Automotive Studies, Tongji University, Shanghai 201804, ChinaInstitute of Intelligent Vehicles, School of Automotive Studies, Tongji University, Shanghai 201804, ChinaInstitute of Intelligent Vehicles, School of Automotive Studies, Tongji University, Shanghai 201804, ChinaInstitute of Intelligent Vehicles, School of Automotive Studies, Tongji University, Shanghai 201804, ChinaThe automated driving of agricultural machinery is of great significance for the agricultural production efficiency, yet is still challenging due to the significantly varied environmental conditions through day and night. To address operation safety for pedestrians in farmland, this paper proposes a 3D person sensing approach based on monocular RGB and Far-Infrared (FIR) images. Since public available datasets for agricultural 3D pedestrian detection are scarce, a new dataset is proposed, named as “FieldSafePedestrian”, which includes field images in both day and night. The implemented data augmentations of night images and semi-automatic labeling approach are also elaborated to facilitate the 3D annotation of pedestrians. To fuse heterogeneous images of sensors with non-parallel optical axis, the Dual-Input Depth-Guided Dynamic-Depthwise-Dilated Fusion network (D5F) is proposed, which assists the pixel alignment between FIR and RGB images with estimated depth information and deploys a dynamic filtering to guide the heterogeneous information fusion. Experiments on field images in both daytime and nighttime demonstrate that compared with the state-of-the-arts, the dynamic aligned image fusion achieves an accuracy gain of 3.9% and 4.5% in terms of center distance and BEV-IOU, respectively, without affecting the run-time efficiency.https://www.mdpi.com/2072-4292/13/15/2896heterogeneous sensor fusionday- and nighttime perceptionagricultural dataset3D pedestrian detection |
| spellingShingle | Wei Tian Zhenwen Deng Dong Yin Zehan Zheng Yuyao Huang Xin Bi 3D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared Sensing Remote Sensing heterogeneous sensor fusion day- and nighttime perception agricultural dataset 3D pedestrian detection |
| title | 3D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared Sensing |
| title_full | 3D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared Sensing |
| title_fullStr | 3D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared Sensing |
| title_full_unstemmed | 3D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared Sensing |
| title_short | 3D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared Sensing |
| title_sort | 3d pedestrian detection in farmland by monocular rgb image and far infrared sensing |
| topic | heterogeneous sensor fusion day- and nighttime perception agricultural dataset 3D pedestrian detection |
| url | https://www.mdpi.com/2072-4292/13/15/2896 |
| work_keys_str_mv | AT weitian 3dpedestriandetectioninfarmlandbymonocularrgbimageandfarinfraredsensing AT zhenwendeng 3dpedestriandetectioninfarmlandbymonocularrgbimageandfarinfraredsensing AT dongyin 3dpedestriandetectioninfarmlandbymonocularrgbimageandfarinfraredsensing AT zehanzheng 3dpedestriandetectioninfarmlandbymonocularrgbimageandfarinfraredsensing AT yuyaohuang 3dpedestriandetectioninfarmlandbymonocularrgbimageandfarinfraredsensing AT xinbi 3dpedestriandetectioninfarmlandbymonocularrgbimageandfarinfraredsensing |