Smart Glass System Using Deep Learning for the Blind and Visually Impaired
Individuals suffering from visual impairments and blindness encounter difficulties in moving independently and overcoming various problems in their routine lives. As a solution, artificial intelligence and computer vision approaches facilitate blind and visually impaired (BVI) people in fulfilling t...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-11-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/10/22/2756 |
| _version_ | 1797510616509841408 |
|---|---|
| author | Mukhriddin Mukhiddinov Jinsoo Cho |
| author_facet | Mukhriddin Mukhiddinov Jinsoo Cho |
| author_sort | Mukhriddin Mukhiddinov |
| collection | DOAJ |
| description | Individuals suffering from visual impairments and blindness encounter difficulties in moving independently and overcoming various problems in their routine lives. As a solution, artificial intelligence and computer vision approaches facilitate blind and visually impaired (BVI) people in fulfilling their primary activities without much dependency on other people. Smart glasses are a potential assistive technology for BVI people to aid in individual travel and provide social comfort and safety. However, practically, the BVI are unable move alone, particularly in dark scenes and at night. In this study we propose a smart glass system for BVI people, employing computer vision techniques and deep learning models, audio feedback, and tactile graphics to facilitate independent movement in a night-time environment. The system is divided into four models: a low-light image enhancement model, an object recognition and audio feedback model, a salient object detection model, and a text-to-speech and tactile graphics generation model. Thus, this system was developed to assist in the following manner: (1) enhancing the contrast of images under low-light conditions employing a two-branch exposure-fusion network; (2) guiding users with audio feedback using a transformer encoder–decoder object detection model that can recognize 133 categories of sound, such as people, animals, cars, etc., and (3) accessing visual information using salient object extraction, text recognition, and refreshable tactile display. We evaluated the performance of the system and achieved competitive performance on the challenging Low-Light and ExDark datasets. |
| first_indexed | 2024-03-10T05:33:41Z |
| format | Article |
| id | doaj.art-d40be9b6b49d41eb8836a0adb2d199e0 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-10T05:33:41Z |
| publishDate | 2021-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-d40be9b6b49d41eb8836a0adb2d199e02023-11-22T23:06:39ZengMDPI AGElectronics2079-92922021-11-011022275610.3390/electronics10222756Smart Glass System Using Deep Learning for the Blind and Visually ImpairedMukhriddin Mukhiddinov0Jinsoo Cho1Department of Computer Engineering, Gachon University, Sujeong-gu, Seongnam-si 13120, KoreaDepartment of Computer Engineering, Gachon University, Sujeong-gu, Seongnam-si 13120, KoreaIndividuals suffering from visual impairments and blindness encounter difficulties in moving independently and overcoming various problems in their routine lives. As a solution, artificial intelligence and computer vision approaches facilitate blind and visually impaired (BVI) people in fulfilling their primary activities without much dependency on other people. Smart glasses are a potential assistive technology for BVI people to aid in individual travel and provide social comfort and safety. However, practically, the BVI are unable move alone, particularly in dark scenes and at night. In this study we propose a smart glass system for BVI people, employing computer vision techniques and deep learning models, audio feedback, and tactile graphics to facilitate independent movement in a night-time environment. The system is divided into four models: a low-light image enhancement model, an object recognition and audio feedback model, a salient object detection model, and a text-to-speech and tactile graphics generation model. Thus, this system was developed to assist in the following manner: (1) enhancing the contrast of images under low-light conditions employing a two-branch exposure-fusion network; (2) guiding users with audio feedback using a transformer encoder–decoder object detection model that can recognize 133 categories of sound, such as people, animals, cars, etc., and (3) accessing visual information using salient object extraction, text recognition, and refreshable tactile display. We evaluated the performance of the system and achieved competitive performance on the challenging Low-Light and ExDark datasets.https://www.mdpi.com/2079-9292/10/22/2756smart glassesartificial intelligenceblind and visually impaireddeep learninglow-light imagesassistive technologies |
| spellingShingle | Mukhriddin Mukhiddinov Jinsoo Cho Smart Glass System Using Deep Learning for the Blind and Visually Impaired Electronics smart glasses artificial intelligence blind and visually impaired deep learning low-light images assistive technologies |
| title | Smart Glass System Using Deep Learning for the Blind and Visually Impaired |
| title_full | Smart Glass System Using Deep Learning for the Blind and Visually Impaired |
| title_fullStr | Smart Glass System Using Deep Learning for the Blind and Visually Impaired |
| title_full_unstemmed | Smart Glass System Using Deep Learning for the Blind and Visually Impaired |
| title_short | Smart Glass System Using Deep Learning for the Blind and Visually Impaired |
| title_sort | smart glass system using deep learning for the blind and visually impaired |
| topic | smart glasses artificial intelligence blind and visually impaired deep learning low-light images assistive technologies |
| url | https://www.mdpi.com/2079-9292/10/22/2756 |
| work_keys_str_mv | AT mukhriddinmukhiddinov smartglasssystemusingdeeplearningfortheblindandvisuallyimpaired AT jinsoocho smartglasssystemusingdeeplearningfortheblindandvisuallyimpaired |