An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos
With the rapid development of unmanned aerial vehicles (UAVs), UAV-based intelligent airborne surveillance systems represented by real-time ground vehicle speed estimation have attracted wide attention from researchers. However, there are still many challenges in extracting speed information from UA...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-05-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/11/10/1241 |
| _version_ | 1818974464171311104 |
|---|---|
| author | Jing Li Shuo Chen Fangbing Zhang Erkang Li Tao Yang Zhaoyang Lu |
| author_facet | Jing Li Shuo Chen Fangbing Zhang Erkang Li Tao Yang Zhaoyang Lu |
| author_sort | Jing Li |
| collection | DOAJ |
| description | With the rapid development of unmanned aerial vehicles (UAVs), UAV-based intelligent airborne surveillance systems represented by real-time ground vehicle speed estimation have attracted wide attention from researchers. However, there are still many challenges in extracting speed information from UAV videos, including the dynamic moving background, small target size, complicated environment, and diverse scenes. In this paper, we propose a novel adaptive framework for multi-vehicle ground speed estimation in airborne videos. Firstly, we build a traffic dataset based on UAV. Then, we use the deep learning detection algorithm to detect the vehicle in the UAV field of view and obtain the trajectory in the image through the tracking-by-detection algorithm. Thereafter, we present a motion compensation method based on homography. This method obtains matching feature points by an optical flow method and eliminates the influence of the detected target to accurately calculate the homography matrix to determine the real motion trajectory in the current frame. Finally, vehicle speed is estimated based on the mapping relationship between the pixel distance and the actual distance. The method regards the actual size of the car as prior information and adaptively recovers the pixel scale by estimating the vehicle size in the image; it then calculates the vehicle speed. In order to evaluate the performance of the proposed system, we carry out a large number of experiments on the AirSim Simulation platform as well as real UAV aerial surveillance experiments. Through quantitative and qualitative analysis of the simulation results and real experiments, we verify that the proposed system has a unique ability to detect, track, and estimate the speed of ground vehicles simultaneously even with a single downward-looking camera. Additionally, the system can obtain effective and accurate speed estimation results, even in various complex scenes. |
| first_indexed | 2024-12-20T15:40:28Z |
| format | Article |
| id | doaj.art-e25089669ea046508b242687b92692b7 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-12-20T15:40:28Z |
| publishDate | 2019-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-e25089669ea046508b242687b92692b72022-12-21T19:35:13ZengMDPI AGRemote Sensing2072-42922019-05-011110124110.3390/rs11101241rs11101241An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne VideosJing Li0Shuo Chen1Fangbing Zhang2Erkang Li3Tao Yang4Zhaoyang Lu5School of Telecommunications Engineering, Xidian University, Xi’an 710000, ChinaSchool of Telecommunications Engineering, Xidian University, Xi’an 710000, ChinaSchool of Computer Science, Northwestern Polytechnical University, Xi’an 710000, ChinaSchool of Telecommunications Engineering, Xidian University, Xi’an 710000, ChinaSchool of Computer Science, Northwestern Polytechnical University, Xi’an 710000, ChinaSchool of Telecommunications Engineering, Xidian University, Xi’an 710000, ChinaWith the rapid development of unmanned aerial vehicles (UAVs), UAV-based intelligent airborne surveillance systems represented by real-time ground vehicle speed estimation have attracted wide attention from researchers. However, there are still many challenges in extracting speed information from UAV videos, including the dynamic moving background, small target size, complicated environment, and diverse scenes. In this paper, we propose a novel adaptive framework for multi-vehicle ground speed estimation in airborne videos. Firstly, we build a traffic dataset based on UAV. Then, we use the deep learning detection algorithm to detect the vehicle in the UAV field of view and obtain the trajectory in the image through the tracking-by-detection algorithm. Thereafter, we present a motion compensation method based on homography. This method obtains matching feature points by an optical flow method and eliminates the influence of the detected target to accurately calculate the homography matrix to determine the real motion trajectory in the current frame. Finally, vehicle speed is estimated based on the mapping relationship between the pixel distance and the actual distance. The method regards the actual size of the car as prior information and adaptively recovers the pixel scale by estimating the vehicle size in the image; it then calculates the vehicle speed. In order to evaluate the performance of the proposed system, we carry out a large number of experiments on the AirSim Simulation platform as well as real UAV aerial surveillance experiments. Through quantitative and qualitative analysis of the simulation results and real experiments, we verify that the proposed system has a unique ability to detect, track, and estimate the speed of ground vehicles simultaneously even with a single downward-looking camera. Additionally, the system can obtain effective and accurate speed estimation results, even in various complex scenes.https://www.mdpi.com/2072-4292/11/10/1241ground vehicle speed estimationintelligent airborne video surveillanceunmanned aerial vehicleobject detection and trackingmotion compensation |
| spellingShingle | Jing Li Shuo Chen Fangbing Zhang Erkang Li Tao Yang Zhaoyang Lu An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos Remote Sensing ground vehicle speed estimation intelligent airborne video surveillance unmanned aerial vehicle object detection and tracking motion compensation |
| title | An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos |
| title_full | An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos |
| title_fullStr | An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos |
| title_full_unstemmed | An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos |
| title_short | An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos |
| title_sort | adaptive framework for multi vehicle ground speed estimation in airborne videos |
| topic | ground vehicle speed estimation intelligent airborne video surveillance unmanned aerial vehicle object detection and tracking motion compensation |
| url | https://www.mdpi.com/2072-4292/11/10/1241 |
| work_keys_str_mv | AT jingli anadaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT shuochen anadaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT fangbingzhang anadaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT erkangli anadaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT taoyang anadaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT zhaoyanglu anadaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT jingli adaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT shuochen adaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT fangbingzhang adaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT erkangli adaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT taoyang adaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos AT zhaoyanglu adaptiveframeworkformultivehiclegroundspeedestimationinairbornevideos |