Multi Visual Modality Fall Detection Dataset
Falls are one of the leading causes of injury-related deaths among the elderly worldwide. Effective detection of falls can reduce the risk of complications and injuries. Fall detection can be performed using wearable devices or ambient sensors; these methods may struggle with user compliance issues...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9910156/ |
| _version_ | 1811240784414900224 |
|---|---|
| author | Stefan Denkovski Shehroz S. Khan Brandon Malamis Sae Young Moon Bing Ye Alex Mihailidis |
| author_facet | Stefan Denkovski Shehroz S. Khan Brandon Malamis Sae Young Moon Bing Ye Alex Mihailidis |
| author_sort | Stefan Denkovski |
| collection | DOAJ |
| description | Falls are one of the leading causes of injury-related deaths among the elderly worldwide. Effective detection of falls can reduce the risk of complications and injuries. Fall detection can be performed using wearable devices or ambient sensors; these methods may struggle with user compliance issues or false alarms. Video cameras provide a passive alternative; however, regular red, green, and blue (RGB) cameras are impacted by changing lighting conditions and privacy concerns. From a machine learning perspective, developing an effective fall detection system is challenging because of the rarity and variability of falls. Many existing fall detection datasets lack important real-world considerations, such as varied lighting, continuous activities of daily living (ADLs), and camera placement. The lack of these considerations makes it difficult to develop predictive models that can operate effectively in the real world. To address these limitations, we introduce a novel multi-modality dataset (MUVIM) that contains four visual modalities: infra-red, depth, RGB and thermal cameras. These modalities offer benefits such as obfuscated facial features and improved performance in low-light conditions. We formulated fall detection as an anomaly detection problem, in which a customized spatio-temporal convolutional autoencoder was trained only on ADLs so that a fall would increase the reconstruction error. Our results showed that infra-red cameras provided the highest level of performance (AUC ROC = 0.94), followed by thermal (AUC ROC = 0.87), depth (AUC ROC = 0.86) and RGB (AUC ROC = 0.83). This research provides a unique opportunity to analyze the utility of camera modalities in detecting falls in a home setting while balancing performance, passiveness, and privacy. |
| first_indexed | 2024-04-12T13:26:28Z |
| format | Article |
| id | doaj.art-8cb4a28d37a14fe5a0fe8e9379ec3b6c |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T13:26:28Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-8cb4a28d37a14fe5a0fe8e9379ec3b6c2022-12-22T03:31:19ZengIEEEIEEE Access2169-35362022-01-011010642210643510.1109/ACCESS.2022.32119399910156Multi Visual Modality Fall Detection DatasetStefan Denkovski0https://orcid.org/0000-0001-9527-7261Shehroz S. Khan1https://orcid.org/0000-0002-1195-4999Brandon Malamis2Sae Young Moon3Bing Ye4Alex Mihailidis5KITE Research Institute, Toronto Rehabilitation Institute, University Health Network, Toronto, ON, CanadaKITE Research Institute, Toronto Rehabilitation Institute, University Health Network, Toronto, ON, CanadaInstitute of Biomedical Engineering, University of Toronto, Toronto, ON, CanadaKITE Research Institute, Toronto Rehabilitation Institute, University Health Network, Toronto, ON, CanadaKITE Research Institute, Toronto Rehabilitation Institute, University Health Network, Toronto, ON, CanadaKITE Research Institute, Toronto Rehabilitation Institute, University Health Network, Toronto, ON, CanadaFalls are one of the leading causes of injury-related deaths among the elderly worldwide. Effective detection of falls can reduce the risk of complications and injuries. Fall detection can be performed using wearable devices or ambient sensors; these methods may struggle with user compliance issues or false alarms. Video cameras provide a passive alternative; however, regular red, green, and blue (RGB) cameras are impacted by changing lighting conditions and privacy concerns. From a machine learning perspective, developing an effective fall detection system is challenging because of the rarity and variability of falls. Many existing fall detection datasets lack important real-world considerations, such as varied lighting, continuous activities of daily living (ADLs), and camera placement. The lack of these considerations makes it difficult to develop predictive models that can operate effectively in the real world. To address these limitations, we introduce a novel multi-modality dataset (MUVIM) that contains four visual modalities: infra-red, depth, RGB and thermal cameras. These modalities offer benefits such as obfuscated facial features and improved performance in low-light conditions. We formulated fall detection as an anomaly detection problem, in which a customized spatio-temporal convolutional autoencoder was trained only on ADLs so that a fall would increase the reconstruction error. Our results showed that infra-red cameras provided the highest level of performance (AUC ROC = 0.94), followed by thermal (AUC ROC = 0.87), depth (AUC ROC = 0.86) and RGB (AUC ROC = 0.83). This research provides a unique opportunity to analyze the utility of camera modalities in detecting falls in a home setting while balancing performance, passiveness, and privacy.https://ieeexplore.ieee.org/document/9910156/Fall detectionmulti-modalautoencoderanomaly detectiondeep learningcomputer vision |
| spellingShingle | Stefan Denkovski Shehroz S. Khan Brandon Malamis Sae Young Moon Bing Ye Alex Mihailidis Multi Visual Modality Fall Detection Dataset IEEE Access Fall detection multi-modal autoencoder anomaly detection deep learning computer vision |
| title | Multi Visual Modality Fall Detection Dataset |
| title_full | Multi Visual Modality Fall Detection Dataset |
| title_fullStr | Multi Visual Modality Fall Detection Dataset |
| title_full_unstemmed | Multi Visual Modality Fall Detection Dataset |
| title_short | Multi Visual Modality Fall Detection Dataset |
| title_sort | multi visual modality fall detection dataset |
| topic | Fall detection multi-modal autoencoder anomaly detection deep learning computer vision |
| url | https://ieeexplore.ieee.org/document/9910156/ |
| work_keys_str_mv | AT stefandenkovski multivisualmodalityfalldetectiondataset AT shehrozskhan multivisualmodalityfalldetectiondataset AT brandonmalamis multivisualmodalityfalldetectiondataset AT saeyoungmoon multivisualmodalityfalldetectiondataset AT bingye multivisualmodalityfalldetectiondataset AT alexmihailidis multivisualmodalityfalldetectiondataset |