DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance
In this paper, we introduce the so-called DEEP-SEE framework that jointly exploits computer vision algorithms and deep convolutional neural networks (CNNs) to detect, track and recognize in real time objects encountered during navigation in the outdoor environment. A first feature concerns an object...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2017-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/17/11/2473 |
| _version_ | 1828348386231713792 |
|---|---|
| author | Ruxandra Tapu Bogdan Mocanu Titus Zaharia |
| author_facet | Ruxandra Tapu Bogdan Mocanu Titus Zaharia |
| author_sort | Ruxandra Tapu |
| collection | DOAJ |
| description | In this paper, we introduce the so-called DEEP-SEE framework that jointly exploits computer vision algorithms and deep convolutional neural networks (CNNs) to detect, track and recognize in real time objects encountered during navigation in the outdoor environment. A first feature concerns an object detection technique designed to localize both static and dynamic objects without any a priori knowledge about their position, type or shape. The methodological core of the proposed approach relies on a novel object tracking method based on two convolutional neural networks trained offline. The key principle consists of alternating between tracking using motion information and predicting the object location in time based on visual similarity. The validation of the tracking technique is performed on standard benchmark VOT datasets, and shows that the proposed approach returns state-of-the-art results while minimizing the computational complexity. Then, the DEEP-SEE framework is integrated into a novel assistive device, designed to improve cognition of VI people and to increase their safety when navigating in crowded urban scenes. The validation of our assistive device is performed on a video dataset with 30 elements acquired with the help of VI users. The proposed system shows high accuracy (>90%) and robustness (>90%) scores regardless on the scene dynamics. |
| first_indexed | 2024-04-14T00:53:15Z |
| format | Article |
| id | doaj.art-095cc302233945ef9317a78959697951 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-14T00:53:15Z |
| publishDate | 2017-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-095cc302233945ef9317a789596979512022-12-22T02:21:42ZengMDPI AGSensors1424-82202017-10-011711247310.3390/s17112473s17112473DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational AssistanceRuxandra Tapu0Bogdan Mocanu1Titus Zaharia2Advanced Research and TEchniques for Multidimensional Imaging Systems Department, Institut Mines-Télécom/Télécom SudParis, UMR CNRS MAP5 8145 and 5157 SAMOVAR, 9 rue Charles Fourier, 91000 Évry, FranceAdvanced Research and TEchniques for Multidimensional Imaging Systems Department, Institut Mines-Télécom/Télécom SudParis, UMR CNRS MAP5 8145 and 5157 SAMOVAR, 9 rue Charles Fourier, 91000 Évry, FranceAdvanced Research and TEchniques for Multidimensional Imaging Systems Department, Institut Mines-Télécom/Télécom SudParis, UMR CNRS MAP5 8145 and 5157 SAMOVAR, 9 rue Charles Fourier, 91000 Évry, FranceIn this paper, we introduce the so-called DEEP-SEE framework that jointly exploits computer vision algorithms and deep convolutional neural networks (CNNs) to detect, track and recognize in real time objects encountered during navigation in the outdoor environment. A first feature concerns an object detection technique designed to localize both static and dynamic objects without any a priori knowledge about their position, type or shape. The methodological core of the proposed approach relies on a novel object tracking method based on two convolutional neural networks trained offline. The key principle consists of alternating between tracking using motion information and predicting the object location in time based on visual similarity. The validation of the tracking technique is performed on standard benchmark VOT datasets, and shows that the proposed approach returns state-of-the-art results while minimizing the computational complexity. Then, the DEEP-SEE framework is integrated into a novel assistive device, designed to improve cognition of VI people and to increase their safety when navigating in crowded urban scenes. The validation of our assistive device is performed on a video dataset with 30 elements acquired with the help of VI users. The proposed system shows high accuracy (>90%) and robustness (>90%) scores regardless on the scene dynamics.https://www.mdpi.com/1424-8220/17/11/2473object detectiontracking and recognitionconvolutional neural networksvisually impaired userswearable assistive device |
| spellingShingle | Ruxandra Tapu Bogdan Mocanu Titus Zaharia DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance Sensors object detection tracking and recognition convolutional neural networks visually impaired users wearable assistive device |
| title | DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance |
| title_full | DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance |
| title_fullStr | DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance |
| title_full_unstemmed | DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance |
| title_short | DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance |
| title_sort | deep see joint object detection tracking and recognition with application to visually impaired navigational assistance |
| topic | object detection tracking and recognition convolutional neural networks visually impaired users wearable assistive device |
| url | https://www.mdpi.com/1424-8220/17/11/2473 |
| work_keys_str_mv | AT ruxandratapu deepseejointobjectdetectiontrackingandrecognitionwithapplicationtovisuallyimpairednavigationalassistance AT bogdanmocanu deepseejointobjectdetectiontrackingandrecognitionwithapplicationtovisuallyimpairednavigationalassistance AT tituszaharia deepseejointobjectdetectiontrackingandrecognitionwithapplicationtovisuallyimpairednavigationalassistance |