Pupil Size Prediction Techniques Based on Convolution Neural Network
The size of one’s pupil can indicate one’s physical condition and mental state. When we search related papers about AI and the pupil, most studies focused on eye-tracking. This paper proposes an algorithm that can calculate pupil size based on a convolution neural network (CNN). Usually, the shape o...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/15/4965 |
| _version_ | 1797525165987332096 |
|---|---|
| author | Allen Jong-Woei Whang Yi-Yung Chen Wei-Chieh Tseng Chih-Hsien Tsai Yi-Ping Chao Chieh-Hung Yen Chun-Hsiu Liu Xin Zhang |
| author_facet | Allen Jong-Woei Whang Yi-Yung Chen Wei-Chieh Tseng Chih-Hsien Tsai Yi-Ping Chao Chieh-Hung Yen Chun-Hsiu Liu Xin Zhang |
| author_sort | Allen Jong-Woei Whang |
| collection | DOAJ |
| description | The size of one’s pupil can indicate one’s physical condition and mental state. When we search related papers about AI and the pupil, most studies focused on eye-tracking. This paper proposes an algorithm that can calculate pupil size based on a convolution neural network (CNN). Usually, the shape of the pupil is not round, and 50% of pupils can be calculated using ellipses as the best fitting shapes. This paper uses the major and minor axes of an ellipse to represent the size of pupils and uses the two parameters as the output of the network. Regarding the input of the network, the dataset is in video format (continuous frames). Taking each frame from the videos and using these to train the CNN model may cause overfitting since the images are too similar. This study used data augmentation and calculated the structural similarity to ensure that the images had a certain degree of difference to avoid this problem. For optimizing the network structure, this study compared the mean error with changes in the depth of the network and the field of view (FOV) of the convolution filter. The result shows that both deepening the network and widening the FOV of the convolution filter can reduce the mean error. According to the results, the mean error of the pupil length is 5.437% and the pupil area is 10.57%. It can operate in low-cost mobile embedded systems at 35 frames per second, demonstrating that low-cost designs can be used for pupil size prediction. |
| first_indexed | 2024-03-10T09:08:56Z |
| format | Article |
| id | doaj.art-2f6976e3470d413f9beee433f2c5b954 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T09:08:56Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-2f6976e3470d413f9beee433f2c5b9542023-11-22T06:08:24ZengMDPI AGSensors1424-82202021-07-012115496510.3390/s21154965Pupil Size Prediction Techniques Based on Convolution Neural NetworkAllen Jong-Woei Whang0Yi-Yung Chen1Wei-Chieh Tseng2Chih-Hsien Tsai3Yi-Ping Chao4Chieh-Hung Yen5Chun-Hsiu Liu6Xin Zhang7Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei City 106335, TaiwanGraduate Institute of Color & Illumination Technology, National Taiwan University of Science and Technology, Taipei City 106335, TaiwanDepartment of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei City 106335, TaiwanGraduate Institute of Electro-Optical Engineering, National Taiwan University of Science and Technology, Taipei City 106335, TaiwanGraduate Institute of Biomedical Engineering, Chang Gung University, Taoyuan City 333323, TaiwanGraduate Institute of Biomedical Engineering, Chang Gung University, Taoyuan City 333323, TaiwanDepartment of Ophthalmology, Chang Gung Memorial Hospital at Linkou, Taoyuan City 333423, TaiwanDepartment of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei City 106335, TaiwanThe size of one’s pupil can indicate one’s physical condition and mental state. When we search related papers about AI and the pupil, most studies focused on eye-tracking. This paper proposes an algorithm that can calculate pupil size based on a convolution neural network (CNN). Usually, the shape of the pupil is not round, and 50% of pupils can be calculated using ellipses as the best fitting shapes. This paper uses the major and minor axes of an ellipse to represent the size of pupils and uses the two parameters as the output of the network. Regarding the input of the network, the dataset is in video format (continuous frames). Taking each frame from the videos and using these to train the CNN model may cause overfitting since the images are too similar. This study used data augmentation and calculated the structural similarity to ensure that the images had a certain degree of difference to avoid this problem. For optimizing the network structure, this study compared the mean error with changes in the depth of the network and the field of view (FOV) of the convolution filter. The result shows that both deepening the network and widening the FOV of the convolution filter can reduce the mean error. According to the results, the mean error of the pupil length is 5.437% and the pupil area is 10.57%. It can operate in low-cost mobile embedded systems at 35 frames per second, demonstrating that low-cost designs can be used for pupil size prediction.https://www.mdpi.com/1424-8220/21/15/4965biomedical imagingcomputational intelligenceengineering in medicine and biologymachine learning |
| spellingShingle | Allen Jong-Woei Whang Yi-Yung Chen Wei-Chieh Tseng Chih-Hsien Tsai Yi-Ping Chao Chieh-Hung Yen Chun-Hsiu Liu Xin Zhang Pupil Size Prediction Techniques Based on Convolution Neural Network Sensors biomedical imaging computational intelligence engineering in medicine and biology machine learning |
| title | Pupil Size Prediction Techniques Based on Convolution Neural Network |
| title_full | Pupil Size Prediction Techniques Based on Convolution Neural Network |
| title_fullStr | Pupil Size Prediction Techniques Based on Convolution Neural Network |
| title_full_unstemmed | Pupil Size Prediction Techniques Based on Convolution Neural Network |
| title_short | Pupil Size Prediction Techniques Based on Convolution Neural Network |
| title_sort | pupil size prediction techniques based on convolution neural network |
| topic | biomedical imaging computational intelligence engineering in medicine and biology machine learning |
| url | https://www.mdpi.com/1424-8220/21/15/4965 |
| work_keys_str_mv | AT allenjongwoeiwhang pupilsizepredictiontechniquesbasedonconvolutionneuralnetwork AT yiyungchen pupilsizepredictiontechniquesbasedonconvolutionneuralnetwork AT weichiehtseng pupilsizepredictiontechniquesbasedonconvolutionneuralnetwork AT chihhsientsai pupilsizepredictiontechniquesbasedonconvolutionneuralnetwork AT yipingchao pupilsizepredictiontechniquesbasedonconvolutionneuralnetwork AT chiehhungyen pupilsizepredictiontechniquesbasedonconvolutionneuralnetwork AT chunhsiuliu pupilsizepredictiontechniquesbasedonconvolutionneuralnetwork AT xinzhang pupilsizepredictiontechniquesbasedonconvolutionneuralnetwork |