Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation
BackgroundEye movement tests remain significantly underutilized in emergency departments and primary healthcare units, despite their superior diagnostic sensitivity compared to neuroimaging modalities for the differential diagnosis of acute vertigo. This underutilization may be attributed to a poten...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-02-01
|
| Series: | Frontiers in Neurology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fneur.2024.1342108/full |
| _version_ | 1797301572580933632 |
|---|---|
| author | Changje Cho Sejik Park Sunmi Ma Hyo-Jeong Lee Hyo-Jeong Lee Eun-Cheon Lim Sung Kwang Hong Sung Kwang Hong |
| author_facet | Changje Cho Sejik Park Sunmi Ma Hyo-Jeong Lee Hyo-Jeong Lee Eun-Cheon Lim Sung Kwang Hong Sung Kwang Hong |
| author_sort | Changje Cho |
| collection | DOAJ |
| description | BackgroundEye movement tests remain significantly underutilized in emergency departments and primary healthcare units, despite their superior diagnostic sensitivity compared to neuroimaging modalities for the differential diagnosis of acute vertigo. This underutilization may be attributed to a potential lack of awareness regarding these tests and the absence of appropriate tools for detecting nystagmus. This study aimed to develop a nystagmus measurement algorithm using a lightweight deep-learning model that recognizes the ocular regions.MethodThe deep learning model was used to segment the eye regions, detect blinking, and determine the pupil center. The model was trained using images extracted from video clips of a clinical battery of eye movement tests and synthesized images reproducing real eye movement scenarios using virtual reality. Each eye image was annotated with segmentation masks of the sclera, iris, and pupil, with gaze vectors of the pupil center for eye tracking. We conducted a comprehensive evaluation of model performance and its execution speeds in comparison to various alternative models using metrics that are suitable for the tasks.ResultsThe mean Intersection over Union values of the segmentation model ranged from 0.90 to 0.97 for different classes (sclera, iris, and pupil) across types of images (synthetic vs. real-world images). Additionally, the mean absolute error for eye tracking was 0.595 for real-world data and the F1 score for blink detection was ≥ 0.95, which indicates our model is performing at a very high level of accuracy. Execution speed was also the most rapid for ocular object segmentation under the same hardware condition as compared to alternative models. The prediction for horizontal and vertical nystagmus in real eye movement video revealed high accuracy with a strong correlation between the observed and predicted values (r = 0.9949 for horizontal and r = 0.9950 for vertical; both p < 0.05).ConclusionThe potential of our model, which can automatically segment ocular regions and track nystagmus in real time from eye movement videos, holds significant promise for emergency settings or remote intervention within the field of neurotology. |
| first_indexed | 2024-03-07T23:24:31Z |
| format | Article |
| id | doaj.art-1f252bd6f22a4a0597244ca5176a1bd0 |
| institution | Directory Open Access Journal |
| issn | 1664-2295 |
| language | English |
| last_indexed | 2024-03-07T23:24:31Z |
| publishDate | 2024-02-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurology |
| spelling | doaj.art-1f252bd6f22a4a0597244ca5176a1bd02024-02-21T05:21:49ZengFrontiers Media S.A.Frontiers in Neurology1664-22952024-02-011510.3389/fneur.2024.13421081342108Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentationChangje Cho0Sejik Park1Sunmi Ma2Hyo-Jeong Lee3Hyo-Jeong Lee4Eun-Cheon Lim5Sung Kwang Hong6Sung Kwang Hong7Hallym University Medical Center, DIDIM Research Institute, Chuncheon, Republic of KoreaHallym University Medical Center, DIDIM Research Institute, Chuncheon, Republic of KoreaDepartment of Otorhinolaryngology-Head and Neck Surgery, Hallym University College of Medicine, Chuncheon, Republic of KoreaHallym University Medical Center, DIDIM Research Institute, Chuncheon, Republic of KoreaDepartment of Otorhinolaryngology-Head and Neck Surgery, Hallym University College of Medicine, Chuncheon, Republic of KoreaHallym University Medical Center, DIDIM Research Institute, Chuncheon, Republic of KoreaHallym University Medical Center, DIDIM Research Institute, Chuncheon, Republic of KoreaDepartment of Otorhinolaryngology-Head and Neck Surgery, Hallym University College of Medicine, Chuncheon, Republic of KoreaBackgroundEye movement tests remain significantly underutilized in emergency departments and primary healthcare units, despite their superior diagnostic sensitivity compared to neuroimaging modalities for the differential diagnosis of acute vertigo. This underutilization may be attributed to a potential lack of awareness regarding these tests and the absence of appropriate tools for detecting nystagmus. This study aimed to develop a nystagmus measurement algorithm using a lightweight deep-learning model that recognizes the ocular regions.MethodThe deep learning model was used to segment the eye regions, detect blinking, and determine the pupil center. The model was trained using images extracted from video clips of a clinical battery of eye movement tests and synthesized images reproducing real eye movement scenarios using virtual reality. Each eye image was annotated with segmentation masks of the sclera, iris, and pupil, with gaze vectors of the pupil center for eye tracking. We conducted a comprehensive evaluation of model performance and its execution speeds in comparison to various alternative models using metrics that are suitable for the tasks.ResultsThe mean Intersection over Union values of the segmentation model ranged from 0.90 to 0.97 for different classes (sclera, iris, and pupil) across types of images (synthetic vs. real-world images). Additionally, the mean absolute error for eye tracking was 0.595 for real-world data and the F1 score for blink detection was ≥ 0.95, which indicates our model is performing at a very high level of accuracy. Execution speed was also the most rapid for ocular object segmentation under the same hardware condition as compared to alternative models. The prediction for horizontal and vertical nystagmus in real eye movement video revealed high accuracy with a strong correlation between the observed and predicted values (r = 0.9949 for horizontal and r = 0.9950 for vertical; both p < 0.05).ConclusionThe potential of our model, which can automatically segment ocular regions and track nystagmus in real time from eye movement videos, holds significant promise for emergency settings or remote intervention within the field of neurotology.https://www.frontiersin.org/articles/10.3389/fneur.2024.1342108/fullvertigoartificial intelligence (AI)CDSSnystagmusfeasibilities studiessegmentation |
| spellingShingle | Changje Cho Sejik Park Sunmi Ma Hyo-Jeong Lee Hyo-Jeong Lee Eun-Cheon Lim Sung Kwang Hong Sung Kwang Hong Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation Frontiers in Neurology vertigo artificial intelligence (AI) CDSS nystagmus feasibilities studies segmentation |
| title | Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation |
| title_full | Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation |
| title_fullStr | Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation |
| title_full_unstemmed | Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation |
| title_short | Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation |
| title_sort | feasibility of video based real time nystagmus tracking a lightweight deep learning model approach using ocular object segmentation |
| topic | vertigo artificial intelligence (AI) CDSS nystagmus feasibilities studies segmentation |
| url | https://www.frontiersin.org/articles/10.3389/fneur.2024.1342108/full |
| work_keys_str_mv | AT changjecho feasibilityofvideobasedrealtimenystagmustrackingalightweightdeeplearningmodelapproachusingocularobjectsegmentation AT sejikpark feasibilityofvideobasedrealtimenystagmustrackingalightweightdeeplearningmodelapproachusingocularobjectsegmentation AT sunmima feasibilityofvideobasedrealtimenystagmustrackingalightweightdeeplearningmodelapproachusingocularobjectsegmentation AT hyojeonglee feasibilityofvideobasedrealtimenystagmustrackingalightweightdeeplearningmodelapproachusingocularobjectsegmentation AT hyojeonglee feasibilityofvideobasedrealtimenystagmustrackingalightweightdeeplearningmodelapproachusingocularobjectsegmentation AT euncheonlim feasibilityofvideobasedrealtimenystagmustrackingalightweightdeeplearningmodelapproachusingocularobjectsegmentation AT sungkwanghong feasibilityofvideobasedrealtimenystagmustrackingalightweightdeeplearningmodelapproachusingocularobjectsegmentation AT sungkwanghong feasibilityofvideobasedrealtimenystagmustrackingalightweightdeeplearningmodelapproachusingocularobjectsegmentation |