Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network
In this paper, we address various challenges in multi-pedestrian and vehicle tracking in high-resolution aerial imagery by intensive evaluation of a number of traditional and Deep Learning based Single- and Multi-Object Tracking methods. We also describe our proposed Deep Learning based Multi-Object...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-05-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/10/1953 |
| _version_ | 1797533768562507776 |
|---|---|
| author | Seyed Majid Azimi Maximilian Kraus Reza Bahmanyar Peter Reinartz |
| author_facet | Seyed Majid Azimi Maximilian Kraus Reza Bahmanyar Peter Reinartz |
| author_sort | Seyed Majid Azimi |
| collection | DOAJ |
| description | In this paper, we address various challenges in multi-pedestrian and vehicle tracking in high-resolution aerial imagery by intensive evaluation of a number of traditional and Deep Learning based Single- and Multi-Object Tracking methods. We also describe our proposed Deep Learning based Multi-Object Tracking method AerialMPTNet that fuses appearance, temporal, and graphical information using a Siamese Neural Network, a Long Short-Term Memory, and a Graph Convolutional Neural Network module for more accurate and stable tracking. Moreover, we investigate the influence of the Squeeze-and-Excitation layers and Online Hard Example Mining on the performance of AerialMPTNet. To the best of our knowledge, we are the first to use these two for regression-based Multi-Object Tracking. Additionally, we studied and compared the <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>L</mi><mn>1</mn></mrow></semantics></math></inline-formula> and Huber loss functions. In our experiments, we extensively evaluate AerialMPTNet on three aerial Multi-Object Tracking datasets, namely AerialMPT and KIT AIS pedestrian and vehicle datasets. Qualitative and quantitative results show that AerialMPTNet outperforms all previous methods for the pedestrian datasets and achieves competitive results for the vehicle dataset. In addition, Long Short-Term Memory and Graph Convolutional Neural Network modules enhance the tracking performance. Moreover, using Squeeze-and-Excitation and Online Hard Example Mining significantly helps for some cases while degrades the results for other cases. In addition, according to the results, <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>L</mi><mn>1</mn></mrow></semantics></math></inline-formula> yields better results with respect to Huber loss for most of the scenarios. The presented results provide a deep insight into challenges and opportunities of the aerial Multi-Object Tracking domain, paving the way for future research. |
| first_indexed | 2024-03-10T11:20:16Z |
| format | Article |
| id | doaj.art-b2ee453fae234112b8cb02d4480a95bf |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T11:20:16Z |
| publishDate | 2021-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-b2ee453fae234112b8cb02d4480a95bf2023-11-21T20:06:46ZengMDPI AGRemote Sensing2072-42922021-05-011310195310.3390/rs13101953Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural NetworkSeyed Majid Azimi0Maximilian Kraus1Reza Bahmanyar2Peter Reinartz3German Aerospace Center (DLR), Remote Sensing Technology Institute (IMF), 82234 Wessling, GermanyDepartment of Informatics, Technical University of Munich, 85748 Garching, GermanyGerman Aerospace Center (DLR), Remote Sensing Technology Institute (IMF), 82234 Wessling, GermanyGerman Aerospace Center (DLR), Remote Sensing Technology Institute (IMF), 82234 Wessling, GermanyIn this paper, we address various challenges in multi-pedestrian and vehicle tracking in high-resolution aerial imagery by intensive evaluation of a number of traditional and Deep Learning based Single- and Multi-Object Tracking methods. We also describe our proposed Deep Learning based Multi-Object Tracking method AerialMPTNet that fuses appearance, temporal, and graphical information using a Siamese Neural Network, a Long Short-Term Memory, and a Graph Convolutional Neural Network module for more accurate and stable tracking. Moreover, we investigate the influence of the Squeeze-and-Excitation layers and Online Hard Example Mining on the performance of AerialMPTNet. To the best of our knowledge, we are the first to use these two for regression-based Multi-Object Tracking. Additionally, we studied and compared the <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>L</mi><mn>1</mn></mrow></semantics></math></inline-formula> and Huber loss functions. In our experiments, we extensively evaluate AerialMPTNet on three aerial Multi-Object Tracking datasets, namely AerialMPT and KIT AIS pedestrian and vehicle datasets. Qualitative and quantitative results show that AerialMPTNet outperforms all previous methods for the pedestrian datasets and achieves competitive results for the vehicle dataset. In addition, Long Short-Term Memory and Graph Convolutional Neural Network modules enhance the tracking performance. Moreover, using Squeeze-and-Excitation and Online Hard Example Mining significantly helps for some cases while degrades the results for other cases. In addition, according to the results, <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>L</mi><mn>1</mn></mrow></semantics></math></inline-formula> yields better results with respect to Huber loss for most of the scenarios. The presented results provide a deep insight into challenges and opportunities of the aerial Multi-Object Tracking domain, paving the way for future research.https://www.mdpi.com/2072-4292/13/10/1953aerial imagerydeep neural networksGraphCNNrecurrent neural networksmulti-object tracking |
| spellingShingle | Seyed Majid Azimi Maximilian Kraus Reza Bahmanyar Peter Reinartz Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network Remote Sensing aerial imagery deep neural networks GraphCNN recurrent neural networks multi-object tracking |
| title | Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network |
| title_full | Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network |
| title_fullStr | Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network |
| title_full_unstemmed | Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network |
| title_short | Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network |
| title_sort | multiple pedestrians and vehicles tracking in aerial imagery using a convolutional neural network |
| topic | aerial imagery deep neural networks GraphCNN recurrent neural networks multi-object tracking |
| url | https://www.mdpi.com/2072-4292/13/10/1953 |
| work_keys_str_mv | AT seyedmajidazimi multiplepedestriansandvehiclestrackinginaerialimageryusingaconvolutionalneuralnetwork AT maximiliankraus multiplepedestriansandvehiclestrackinginaerialimageryusingaconvolutionalneuralnetwork AT rezabahmanyar multiplepedestriansandvehiclestrackinginaerialimageryusingaconvolutionalneuralnetwork AT peterreinartz multiplepedestriansandvehiclestrackinginaerialimageryusingaconvolutionalneuralnetwork |