Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning1
Aerial imagery surveys are commonly used in marine mammal research to determine population size, distribution and habitat use. Analysis of aerial photos involves hours of manually identifying individuals present in each image and converting raw counts into useable biological statistics. Our research...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Canadian Science Publishing
2022-01-01
|
| Series: | Drone Systems and Applications |
| Subjects: | |
| Online Access: | https://cdnsciencepub.com/doi/10.1139/juvs-2021-0024 |
| _version_ | 1797723979855691776 |
|---|---|
| author | Madison L. Harasyn Wayne S. Chan Emma L. Ausen David G. Barber |
| author_facet | Madison L. Harasyn Wayne S. Chan Emma L. Ausen David G. Barber |
| author_sort | Madison L. Harasyn |
| collection | DOAJ |
| description | Aerial imagery surveys are commonly used in marine mammal research to determine population size, distribution and habitat use. Analysis of aerial photos involves hours of manually identifying individuals present in each image and converting raw counts into useable biological statistics. Our research proposes the use of deep learning algorithms to increase the efficiency of the marine mammal research workflow. To test the feasibility of this proposal, the existing YOLOv4 convolutional neural network model was trained to detect belugas, kayaks and motorized boats in oblique drone imagery, collected from a stationary tethered system. Automated computer-based object detection achieved the following precision and recall, respectively, for each class: beluga = 74%/72%; boat = 97%/99%; and kayak = 96%/96%. We then tested the performance of computer vision tracking of belugas and occupied watercraft in drone videos using the DeepSORT tracking algorithm, which achieved a multiple-object tracking accuracy (MOTA) ranging from 37% to 88% and multiple object tracking precision (MOTP) between 63% and 86%. Results from this research indicate that deep learning technology can detect and track features more consistently than human annotators, allowing for larger datasets to be processed within a fraction of the time while avoiding discrepancies introduced by labeling fatigue or multiple human annotators. |
| first_indexed | 2024-03-12T10:10:39Z |
| format | Article |
| id | doaj.art-780718c15a3b4e60b3a13160d795ed8b |
| institution | Directory Open Access Journal |
| issn | 2564-4939 |
| language | English |
| last_indexed | 2024-03-12T10:10:39Z |
| publishDate | 2022-01-01 |
| publisher | Canadian Science Publishing |
| record_format | Article |
| series | Drone Systems and Applications |
| spelling | doaj.art-780718c15a3b4e60b3a13160d795ed8b2023-09-02T10:52:41ZengCanadian Science PublishingDrone Systems and Applications2564-49392022-01-01101779610.1139/juvs-2021-0024Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning1Madison L. Harasyn0Wayne S. Chan1Emma L. Ausen2David G. Barber3Centre for Earth Observation Science, Department of Environment and Geography, University of Manitoba, Winnipeg, CanadaCentre for Earth Observation Science, Department of Environment and Geography, University of Manitoba, Winnipeg, CanadaCentre for Earth Observation Science, Department of Environment and Geography, University of Manitoba, Winnipeg, CanadaCentre for Earth Observation Science, Department of Environment and Geography, University of Manitoba, Winnipeg, CanadaAerial imagery surveys are commonly used in marine mammal research to determine population size, distribution and habitat use. Analysis of aerial photos involves hours of manually identifying individuals present in each image and converting raw counts into useable biological statistics. Our research proposes the use of deep learning algorithms to increase the efficiency of the marine mammal research workflow. To test the feasibility of this proposal, the existing YOLOv4 convolutional neural network model was trained to detect belugas, kayaks and motorized boats in oblique drone imagery, collected from a stationary tethered system. Automated computer-based object detection achieved the following precision and recall, respectively, for each class: beluga = 74%/72%; boat = 97%/99%; and kayak = 96%/96%. We then tested the performance of computer vision tracking of belugas and occupied watercraft in drone videos using the DeepSORT tracking algorithm, which achieved a multiple-object tracking accuracy (MOTA) ranging from 37% to 88% and multiple object tracking precision (MOTP) between 63% and 86%. Results from this research indicate that deep learning technology can detect and track features more consistently than human annotators, allowing for larger datasets to be processed within a fraction of the time while avoiding discrepancies introduced by labeling fatigue or multiple human annotators.https://cdnsciencepub.com/doi/10.1139/juvs-2021-0024computer visiondeep learningunmanned aerial vehicle (UAV)belugaobject detectionobject tracking |
| spellingShingle | Madison L. Harasyn Wayne S. Chan Emma L. Ausen David G. Barber Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning1 Drone Systems and Applications computer vision deep learning unmanned aerial vehicle (UAV) beluga object detection object tracking |
| title | Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning1 |
| title_full | Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning1 |
| title_fullStr | Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning1 |
| title_full_unstemmed | Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning1 |
| title_short | Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning1 |
| title_sort | detection and tracking of belugas kayaks and motorized boats in drone video using deep learning1 |
| topic | computer vision deep learning unmanned aerial vehicle (UAV) beluga object detection object tracking |
| url | https://cdnsciencepub.com/doi/10.1139/juvs-2021-0024 |
| work_keys_str_mv | AT madisonlharasyn detectionandtrackingofbelugaskayaksandmotorizedboatsindronevideousingdeeplearning1 AT wayneschan detectionandtrackingofbelugaskayaksandmotorizedboatsindronevideousingdeeplearning1 AT emmalausen detectionandtrackingofbelugaskayaksandmotorizedboatsindronevideousingdeeplearning1 AT davidgbarber detectionandtrackingofbelugaskayaksandmotorizedboatsindronevideousingdeeplearning1 |