Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation
Abstract Blindness is a main threat that affects the daily life activities of any human. Visual prostheses have been introduced to provide artificial vision to the blind with the aim of allowing them to restore confidence and independence. In this article, we propose an approach that involves four i...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-12-01
|
| Series: | BioMedical Engineering OnLine |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12938-022-01059-7 |
| _version_ | 1797977348510842880 |
|---|---|
| author | Reham H. Elnabawy Slim Abdennadher Olaf Hellwich Seif Eldawlatly |
| author_facet | Reham H. Elnabawy Slim Abdennadher Olaf Hellwich Seif Eldawlatly |
| author_sort | Reham H. Elnabawy |
| collection | DOAJ |
| description | Abstract Blindness is a main threat that affects the daily life activities of any human. Visual prostheses have been introduced to provide artificial vision to the blind with the aim of allowing them to restore confidence and independence. In this article, we propose an approach that involves four image enhancement techniques to facilitate object recognition and localization for visual prostheses users. These techniques are clip art representation of the objects, edge sharpening, corner enhancement and electrode dropout handling. The proposed techniques are tested in a real-time mixed reality simulation environment that mimics vision perceived by visual prostheses users. Twelve experiments were conducted to measure the performance of the participants in object recognition and localization. The experiments involved single objects, multiple objects and navigation. To evaluate the performance of the participants in objects recognition, we measure their recognition time, recognition accuracy and confidence level. For object localization, two metrics were used to measure the performance of the participants which are the grasping attempt time and the grasping accuracy. The results demonstrate that using all enhancement techniques simultaneously gives higher accuracy, higher confidence level and less time for recognizing and grasping objects in comparison to not applying the enhancement techniques or applying pair-wise combinations of them. Visual prostheses could benefit from the proposed approach to provide users with an enhanced perception. |
| first_indexed | 2024-04-11T05:05:23Z |
| format | Article |
| id | doaj.art-8d7a2f5cafb54f42aa513c9ba3a2e5e8 |
| institution | Directory Open Access Journal |
| issn | 1475-925X |
| language | English |
| last_indexed | 2024-04-11T05:05:23Z |
| publishDate | 2022-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BioMedical Engineering OnLine |
| spelling | doaj.art-8d7a2f5cafb54f42aa513c9ba3a2e5e82022-12-25T12:23:33ZengBMCBioMedical Engineering OnLine1475-925X2022-12-0121112510.1186/s12938-022-01059-7Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulationReham H. Elnabawy0Slim Abdennadher1Olaf Hellwich2Seif Eldawlatly3Digital Media Engineering and Technology Department, Faculty of Media Engineering and Technology, German University in CairoComputer Science and Engineering Department, Faculty of Media Engineering and Technology, German University in CairoChair of Computer Vision and Remote Sensing, Technische Universität BerlinComputer and Systems Engineering Department, Faculty of Engineering, Ain Shams UniversityAbstract Blindness is a main threat that affects the daily life activities of any human. Visual prostheses have been introduced to provide artificial vision to the blind with the aim of allowing them to restore confidence and independence. In this article, we propose an approach that involves four image enhancement techniques to facilitate object recognition and localization for visual prostheses users. These techniques are clip art representation of the objects, edge sharpening, corner enhancement and electrode dropout handling. The proposed techniques are tested in a real-time mixed reality simulation environment that mimics vision perceived by visual prostheses users. Twelve experiments were conducted to measure the performance of the participants in object recognition and localization. The experiments involved single objects, multiple objects and navigation. To evaluate the performance of the participants in objects recognition, we measure their recognition time, recognition accuracy and confidence level. For object localization, two metrics were used to measure the performance of the participants which are the grasping attempt time and the grasping accuracy. The results demonstrate that using all enhancement techniques simultaneously gives higher accuracy, higher confidence level and less time for recognizing and grasping objects in comparison to not applying the enhancement techniques or applying pair-wise combinations of them. Visual prostheses could benefit from the proposed approach to provide users with an enhanced perception.https://doi.org/10.1186/s12938-022-01059-7Simulated prosthetic visionObject recognitionObject localizationReal-time mixed reality simulation |
| spellingShingle | Reham H. Elnabawy Slim Abdennadher Olaf Hellwich Seif Eldawlatly Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation BioMedical Engineering OnLine Simulated prosthetic vision Object recognition Object localization Real-time mixed reality simulation |
| title | Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation |
| title_full | Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation |
| title_fullStr | Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation |
| title_full_unstemmed | Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation |
| title_short | Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation |
| title_sort | object recognition and localization enhancement in visual prostheses a real time mixed reality simulation |
| topic | Simulated prosthetic vision Object recognition Object localization Real-time mixed reality simulation |
| url | https://doi.org/10.1186/s12938-022-01059-7 |
| work_keys_str_mv | AT rehamhelnabawy objectrecognitionandlocalizationenhancementinvisualprosthesesarealtimemixedrealitysimulation AT slimabdennadher objectrecognitionandlocalizationenhancementinvisualprosthesesarealtimemixedrealitysimulation AT olafhellwich objectrecognitionandlocalizationenhancementinvisualprosthesesarealtimemixedrealitysimulation AT seifeldawlatly objectrecognitionandlocalizationenhancementinvisualprosthesesarealtimemixedrealitysimulation |