Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case
Multi-human detection and tracking in indoor surveillance is a challenging task due to various factors such as occlusions, illumination changes, and complex human-human and human-object interactions. In this study, we address these challenges by exploring the benefits of a low-level sensor fusion ap...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/8/3993 |
| _version_ | 1797603486604460032 |
|---|---|
| author | Ali Abbasi Sandro Queirós Nuno M. C. da Costa Jaime C. Fonseca João Borges |
| author_facet | Ali Abbasi Sandro Queirós Nuno M. C. da Costa Jaime C. Fonseca João Borges |
| author_sort | Ali Abbasi |
| collection | DOAJ |
| description | Multi-human detection and tracking in indoor surveillance is a challenging task due to various factors such as occlusions, illumination changes, and complex human-human and human-object interactions. In this study, we address these challenges by exploring the benefits of a low-level sensor fusion approach that combines grayscale and neuromorphic vision sensor (NVS) data. We first generate a custom dataset using an NVS camera in an indoor environment. We then conduct a comprehensive study by experimenting with different image features and deep learning networks, followed by a multi-input fusion strategy to optimize our experiments with respect to overfitting. Our primary goal is to determine the best input feature types for multi-human motion detection using statistical analysis. We find that there is a significant difference between the input features of optimized backbones, with the best strategy depending on the amount of available data. Specifically, under a low-data regime, event-based frames seem to be the preferred input feature type, while higher data availability benefits the combined use of grayscale and optical flow features. Our results demonstrate the potential of sensor fusion and deep learning techniques for multi-human tracking in indoor surveillance, although it is acknowledged that further studies are needed to confirm our findings. |
| first_indexed | 2024-03-11T04:32:49Z |
| format | Article |
| id | doaj.art-35681c993a6747f69d553e217fdf0051 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T04:32:49Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-35681c993a6747f69d553e217fdf00512023-11-17T21:17:36ZengMDPI AGSensors1424-82202023-04-01238399310.3390/s23083993Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-CaseAli Abbasi0Sandro Queirós1Nuno M. C. da Costa2Jaime C. Fonseca3João Borges4Algorithmic Center, University of Minho, 4800-058 Azurém, PortugalSchool of Medicine, University of Minho, 4710-057 Gualtar, PortugalAlgorithmic Center, University of Minho, 4800-058 Azurém, PortugalAlgorithmic Center, University of Minho, 4800-058 Azurém, PortugalAlgorithmic Center, University of Minho, 4800-058 Azurém, PortugalMulti-human detection and tracking in indoor surveillance is a challenging task due to various factors such as occlusions, illumination changes, and complex human-human and human-object interactions. In this study, we address these challenges by exploring the benefits of a low-level sensor fusion approach that combines grayscale and neuromorphic vision sensor (NVS) data. We first generate a custom dataset using an NVS camera in an indoor environment. We then conduct a comprehensive study by experimenting with different image features and deep learning networks, followed by a multi-input fusion strategy to optimize our experiments with respect to overfitting. Our primary goal is to determine the best input feature types for multi-human motion detection using statistical analysis. We find that there is a significant difference between the input features of optimized backbones, with the best strategy depending on the amount of available data. Specifically, under a low-data regime, event-based frames seem to be the preferred input feature type, while higher data availability benefits the combined use of grayscale and optical flow features. Our results demonstrate the potential of sensor fusion and deep learning techniques for multi-human tracking in indoor surveillance, although it is acknowledged that further studies are needed to confirm our findings.https://www.mdpi.com/1424-8220/23/8/3993neuromorphic vision sensormultiple human motion detection and trackingmulti-modal datasensor fusionindoor surveillanceevent-based data |
| spellingShingle | Ali Abbasi Sandro Queirós Nuno M. C. da Costa Jaime C. Fonseca João Borges Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case Sensors neuromorphic vision sensor multiple human motion detection and tracking multi-modal data sensor fusion indoor surveillance event-based data |
| title | Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case |
| title_full | Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case |
| title_fullStr | Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case |
| title_full_unstemmed | Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case |
| title_short | Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case |
| title_sort | sensor fusion approach for multiple human motion detection for indoor surveillance use case |
| topic | neuromorphic vision sensor multiple human motion detection and tracking multi-modal data sensor fusion indoor surveillance event-based data |
| url | https://www.mdpi.com/1424-8220/23/8/3993 |
| work_keys_str_mv | AT aliabbasi sensorfusionapproachformultiplehumanmotiondetectionforindoorsurveillanceusecase AT sandroqueiros sensorfusionapproachformultiplehumanmotiondetectionforindoorsurveillanceusecase AT nunomcdacosta sensorfusionapproachformultiplehumanmotiondetectionforindoorsurveillanceusecase AT jaimecfonseca sensorfusionapproachformultiplehumanmotiondetectionforindoorsurveillanceusecase AT joaoborges sensorfusionapproachformultiplehumanmotiondetectionforindoorsurveillanceusecase |