A Smart Context-Aware Hazard Attention System to Help People with Peripheral Vision Loss
Peripheral vision loss results in the inability to detect objects in the peripheral visual field which affects the ability to evaluate and avoid potential hazards. A different number of assistive navigation systems have been developed to help people with vision impairments using wearable and portabl...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/19/7/1630 |
| _version_ | 1798043662150533120 |
|---|---|
| author | Ola Younis Waleed Al-Nuaimy Fiona Rowe Mohammad H. Alomari |
| author_facet | Ola Younis Waleed Al-Nuaimy Fiona Rowe Mohammad H. Alomari |
| author_sort | Ola Younis |
| collection | DOAJ |
| description | Peripheral vision loss results in the inability to detect objects in the peripheral visual field which affects the ability to evaluate and avoid potential hazards. A different number of assistive navigation systems have been developed to help people with vision impairments using wearable and portable devices. Most of these systems are designed to search for obstacles and provide safe navigation paths for visually impaired people without any prioritisation of the degree of danger for each hazard. This paper presents a new context-aware hybrid (indoor/outdoor) hazard classification assistive technology to help people with peripheral vision loss in their navigation using computer-enabled smart glasses equipped with a wide-angle camera. Our proposed system augments users’ existing healthy vision with suitable, meaningful and smart notifications to attract the user’s attention to possible obstructions or hazards in their peripheral field of view. A deep learning object detector is implemented to recognise static and moving objects in real time. After detecting the objects, a Kalman Filter multi-object tracker is used to track these objects over time to determine the motion model. For each tracked object, its motion model represents its way of moving around the user. Motion features are extracted while the object is still in the user’s field of vision. These features are then used to quantify the danger using five predefined hazard classes using a neural network-based classifier. The classification performance is tested on both publicly available and private datasets and the system shows promising results with up to 90% True Positive Rate (TPR) associated with as low as 7% False Positive Rate (FPR), 13% False Negative Rate (FNR) and an average testing Mean Square Error (MSE) of 8.8%. The provided hazard type is then translated into a smart notification to increase the user’s cognitive perception using the healthy vision within the visual field. A participant study was conducted with a group of patients with different visual field defects to explore their feedback about the proposed system and the notification generation stage. The real-world outdoor evaluation of human subjects is planned to be performed in our near future work. |
| first_indexed | 2024-04-11T22:52:14Z |
| format | Article |
| id | doaj.art-dfe919352c9c4a8ba479a755640cc474 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-11T22:52:14Z |
| publishDate | 2019-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-dfe919352c9c4a8ba479a755640cc4742022-12-22T03:58:33ZengMDPI AGSensors1424-82202019-04-01197163010.3390/s19071630s19071630A Smart Context-Aware Hazard Attention System to Help People with Peripheral Vision LossOla Younis0Waleed Al-Nuaimy1Fiona Rowe2Mohammad H. Alomari3Department of Electrical Engineering and Electronics, University of Liverpool, Liverpool L69 3GJ, UKDepartment of Electrical Engineering and Electronics, University of Liverpool, Liverpool L69 3GJ, UKDepartment of Health Services Research, University of Liverpool, Liverpool L69 3GL, UKDepartment of Computer Science, University of Liverpool, Liverpool L69 3BX, UKPeripheral vision loss results in the inability to detect objects in the peripheral visual field which affects the ability to evaluate and avoid potential hazards. A different number of assistive navigation systems have been developed to help people with vision impairments using wearable and portable devices. Most of these systems are designed to search for obstacles and provide safe navigation paths for visually impaired people without any prioritisation of the degree of danger for each hazard. This paper presents a new context-aware hybrid (indoor/outdoor) hazard classification assistive technology to help people with peripheral vision loss in their navigation using computer-enabled smart glasses equipped with a wide-angle camera. Our proposed system augments users’ existing healthy vision with suitable, meaningful and smart notifications to attract the user’s attention to possible obstructions or hazards in their peripheral field of view. A deep learning object detector is implemented to recognise static and moving objects in real time. After detecting the objects, a Kalman Filter multi-object tracker is used to track these objects over time to determine the motion model. For each tracked object, its motion model represents its way of moving around the user. Motion features are extracted while the object is still in the user’s field of vision. These features are then used to quantify the danger using five predefined hazard classes using a neural network-based classifier. The classification performance is tested on both publicly available and private datasets and the system shows promising results with up to 90% True Positive Rate (TPR) associated with as low as 7% False Positive Rate (FPR), 13% False Negative Rate (FNR) and an average testing Mean Square Error (MSE) of 8.8%. The provided hazard type is then translated into a smart notification to increase the user’s cognitive perception using the healthy vision within the visual field. A participant study was conducted with a group of patients with different visual field defects to explore their feedback about the proposed system and the notification generation stage. The real-world outdoor evaluation of human subjects is planned to be performed in our near future work.https://www.mdpi.com/1424-8220/19/7/1630hazard detectioncontext-awarewearable assistive technologyobject detectionobject trackinghazard perceptionaugmented realityKalman filter |
| spellingShingle | Ola Younis Waleed Al-Nuaimy Fiona Rowe Mohammad H. Alomari A Smart Context-Aware Hazard Attention System to Help People with Peripheral Vision Loss Sensors hazard detection context-aware wearable assistive technology object detection object tracking hazard perception augmented reality Kalman filter |
| title | A Smart Context-Aware Hazard Attention System to Help People with Peripheral Vision Loss |
| title_full | A Smart Context-Aware Hazard Attention System to Help People with Peripheral Vision Loss |
| title_fullStr | A Smart Context-Aware Hazard Attention System to Help People with Peripheral Vision Loss |
| title_full_unstemmed | A Smart Context-Aware Hazard Attention System to Help People with Peripheral Vision Loss |
| title_short | A Smart Context-Aware Hazard Attention System to Help People with Peripheral Vision Loss |
| title_sort | smart context aware hazard attention system to help people with peripheral vision loss |
| topic | hazard detection context-aware wearable assistive technology object detection object tracking hazard perception augmented reality Kalman filter |
| url | https://www.mdpi.com/1424-8220/19/7/1630 |
| work_keys_str_mv | AT olayounis asmartcontextawarehazardattentionsystemtohelppeoplewithperipheralvisionloss AT waleedalnuaimy asmartcontextawarehazardattentionsystemtohelppeoplewithperipheralvisionloss AT fionarowe asmartcontextawarehazardattentionsystemtohelppeoplewithperipheralvisionloss AT mohammadhalomari asmartcontextawarehazardattentionsystemtohelppeoplewithperipheralvisionloss AT olayounis smartcontextawarehazardattentionsystemtohelppeoplewithperipheralvisionloss AT waleedalnuaimy smartcontextawarehazardattentionsystemtohelppeoplewithperipheralvisionloss AT fionarowe smartcontextawarehazardattentionsystemtohelppeoplewithperipheralvisionloss AT mohammadhalomari smartcontextawarehazardattentionsystemtohelppeoplewithperipheralvisionloss |