Deep Learning Empowered Wearable-Based Behavior Recognition for Search and Rescue Dogs
Search and Rescue (SaR) dogs are important assets in the hands of first responders, as they have the ability to locate the victim even in cases where the vision and or the sound is limited, due to their inherent talents in olfactory and auditory senses. In this work, we propose a deep-learning-assis...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/3/993 |
| _version_ | 1797484706728509440 |
|---|---|
| author | Panagiotis Kasnesis Vasileios Doulgerakis Dimitris Uzunidis Dimitris G. Kogias Susana I. Funcia Marta B. González Christos Giannousis Charalampos Z. Patrikakis |
| author_facet | Panagiotis Kasnesis Vasileios Doulgerakis Dimitris Uzunidis Dimitris G. Kogias Susana I. Funcia Marta B. González Christos Giannousis Charalampos Z. Patrikakis |
| author_sort | Panagiotis Kasnesis |
| collection | DOAJ |
| description | Search and Rescue (SaR) dogs are important assets in the hands of first responders, as they have the ability to locate the victim even in cases where the vision and or the sound is limited, due to their inherent talents in olfactory and auditory senses. In this work, we propose a deep-learning-assisted implementation incorporating a wearable device, a base station, a mobile application, and a cloud-based infrastructure that can first monitor in real-time the activity, the audio signals, and the location of a SaR dog, and second, recognize and alert the rescuing team whenever the SaR dog spots a victim. For this purpose, we employed deep Convolutional Neural Networks (CNN) both for the activity recognition and the sound classification, which are trained using data from inertial sensors, such as 3-axial accelerometer and gyroscope and from the wearable’s microphone, respectively. The developed deep learning models were deployed on the wearable device, while the overall proposed implementation was validated in two discrete search and rescue scenarios, managing to successfully spot the victim (i.e., obtained F1-score more than 99%) and inform the rescue team in real-time for both scenarios. |
| first_indexed | 2024-03-09T23:09:16Z |
| format | Article |
| id | doaj.art-dfaf81899aee473b8501e98acf124a44 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T23:09:16Z |
| publishDate | 2022-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-dfaf81899aee473b8501e98acf124a442023-11-23T17:48:48ZengMDPI AGSensors1424-82202022-01-0122399310.3390/s22030993Deep Learning Empowered Wearable-Based Behavior Recognition for Search and Rescue DogsPanagiotis Kasnesis0Vasileios Doulgerakis1Dimitris Uzunidis2Dimitris G. Kogias3Susana I. Funcia4Marta B. González5Christos Giannousis6Charalampos Z. Patrikakis7Department of Electrical and Electronic Engineering, University of West Attica, 12244 Athens, GreeceDepartment of Electrical and Electronic Engineering, University of West Attica, 12244 Athens, GreeceDepartment of Electrical and Electronic Engineering, University of West Attica, 12244 Athens, GreeceDepartment of Electrical and Electronic Engineering, University of West Attica, 12244 Athens, GreeceSpanish School of Rescue and Detection with Dogs (ESDP), 28524 Madrid, SpainSpanish School of Rescue and Detection with Dogs (ESDP), 28524 Madrid, SpainDepartment of Electrical and Electronic Engineering, University of West Attica, 12244 Athens, GreeceDepartment of Electrical and Electronic Engineering, University of West Attica, 12244 Athens, GreeceSearch and Rescue (SaR) dogs are important assets in the hands of first responders, as they have the ability to locate the victim even in cases where the vision and or the sound is limited, due to their inherent talents in olfactory and auditory senses. In this work, we propose a deep-learning-assisted implementation incorporating a wearable device, a base station, a mobile application, and a cloud-based infrastructure that can first monitor in real-time the activity, the audio signals, and the location of a SaR dog, and second, recognize and alert the rescuing team whenever the SaR dog spots a victim. For this purpose, we employed deep Convolutional Neural Networks (CNN) both for the activity recognition and the sound classification, which are trained using data from inertial sensors, such as 3-axial accelerometer and gyroscope and from the wearable’s microphone, respectively. The developed deep learning models were deployed on the wearable device, while the overall proposed implementation was validated in two discrete search and rescue scenarios, managing to successfully spot the victim (i.e., obtained F1-score more than 99%) and inform the rescue team in real-time for both scenarios.https://www.mdpi.com/1424-8220/22/3/993deep learningcanine activity recognitionbark detectionwearable computingsearch and rescue system |
| spellingShingle | Panagiotis Kasnesis Vasileios Doulgerakis Dimitris Uzunidis Dimitris G. Kogias Susana I. Funcia Marta B. González Christos Giannousis Charalampos Z. Patrikakis Deep Learning Empowered Wearable-Based Behavior Recognition for Search and Rescue Dogs Sensors deep learning canine activity recognition bark detection wearable computing search and rescue system |
| title | Deep Learning Empowered Wearable-Based Behavior Recognition for Search and Rescue Dogs |
| title_full | Deep Learning Empowered Wearable-Based Behavior Recognition for Search and Rescue Dogs |
| title_fullStr | Deep Learning Empowered Wearable-Based Behavior Recognition for Search and Rescue Dogs |
| title_full_unstemmed | Deep Learning Empowered Wearable-Based Behavior Recognition for Search and Rescue Dogs |
| title_short | Deep Learning Empowered Wearable-Based Behavior Recognition for Search and Rescue Dogs |
| title_sort | deep learning empowered wearable based behavior recognition for search and rescue dogs |
| topic | deep learning canine activity recognition bark detection wearable computing search and rescue system |
| url | https://www.mdpi.com/1424-8220/22/3/993 |
| work_keys_str_mv | AT panagiotiskasnesis deeplearningempoweredwearablebasedbehaviorrecognitionforsearchandrescuedogs AT vasileiosdoulgerakis deeplearningempoweredwearablebasedbehaviorrecognitionforsearchandrescuedogs AT dimitrisuzunidis deeplearningempoweredwearablebasedbehaviorrecognitionforsearchandrescuedogs AT dimitrisgkogias deeplearningempoweredwearablebasedbehaviorrecognitionforsearchandrescuedogs AT susanaifuncia deeplearningempoweredwearablebasedbehaviorrecognitionforsearchandrescuedogs AT martabgonzalez deeplearningempoweredwearablebasedbehaviorrecognitionforsearchandrescuedogs AT christosgiannousis deeplearningempoweredwearablebasedbehaviorrecognitionforsearchandrescuedogs AT charalamposzpatrikakis deeplearningempoweredwearablebasedbehaviorrecognitionforsearchandrescuedogs |