Convolutional Neural Networks Cascade for Automatic Pupil and Iris Detection in Ocular Proton Therapy
Eye tracking techniques based on deep learning are rapidly spreading in a wide variety of application fields. With this study, we want to exploit the potentiality of eye tracking techniques in ocular proton therapy (OPT) applications. We implemented a fully automatic approach based on two-stage conv...
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/13/4400 |
| _version_ | 1797528628755431424 |
|---|---|
| author | Luca Antonioli Andrea Pella Rosalinda Ricotti Matteo Rossi Maria Rosaria Fiore Gabriele Belotti Giuseppe Magro Chiara Paganelli Ester Orlandi Mario Ciocca Guido Baroni |
| author_facet | Luca Antonioli Andrea Pella Rosalinda Ricotti Matteo Rossi Maria Rosaria Fiore Gabriele Belotti Giuseppe Magro Chiara Paganelli Ester Orlandi Mario Ciocca Guido Baroni |
| author_sort | Luca Antonioli |
| collection | DOAJ |
| description | Eye tracking techniques based on deep learning are rapidly spreading in a wide variety of application fields. With this study, we want to exploit the potentiality of eye tracking techniques in ocular proton therapy (OPT) applications. We implemented a fully automatic approach based on two-stage convolutional neural networks (CNNs): the first stage roughly identifies the eye position and the second one performs a fine iris and pupil detection. We selected 707 video frames recorded during clinical operations during OPT treatments performed at our institute. 650 frames were used for training and 57 for a blind test. The estimations of iris and pupil were evaluated against the manual labelled contours delineated by a clinical operator. For iris and pupil predictions, Dice coefficient (median = 0.94 and 0.97), Szymkiewicz–Simpson coefficient (median = 0.97 and 0.98), Intersection over Union coefficient (median = 0.88 and 0.94) and Hausdorff distance (median = 11.6 and 5.0 (pixels)) were quantified. Iris and pupil regions were found to be comparable to the manually labelled ground truths. Our proposed framework could provide an automatic approach to quantitatively evaluating pupil and iris misalignments, and it could be used as an additional support tool for clinical activity, without impacting in any way with the consolidated routine. |
| first_indexed | 2024-03-10T10:01:04Z |
| format | Article |
| id | doaj.art-d47ef6be08624c929ea2fb69a80cf16f |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T10:01:04Z |
| publishDate | 2021-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-d47ef6be08624c929ea2fb69a80cf16f2023-11-22T01:56:54ZengMDPI AGSensors1424-82202021-06-012113440010.3390/s21134400Convolutional Neural Networks Cascade for Automatic Pupil and Iris Detection in Ocular Proton TherapyLuca Antonioli0Andrea Pella1Rosalinda Ricotti2Matteo Rossi3Maria Rosaria Fiore4Gabriele Belotti5Giuseppe Magro6Chiara Paganelli7Ester Orlandi8Mario Ciocca9Guido Baroni10Bioengineering Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, ItalyBioengineering Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, ItalyBioengineering Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, ItalyDepartment of Electronics, Information and Bioengineering, Politecnico di Milano University, 20133 Milan, ItalyRadiotherapy Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, ItalyDepartment of Electronics, Information and Bioengineering, Politecnico di Milano University, 20133 Milan, ItalyMedical Physics Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, ItalyDepartment of Electronics, Information and Bioengineering, Politecnico di Milano University, 20133 Milan, ItalyRadiotherapy Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, ItalyMedical Physics Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, ItalyBioengineering Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, ItalyEye tracking techniques based on deep learning are rapidly spreading in a wide variety of application fields. With this study, we want to exploit the potentiality of eye tracking techniques in ocular proton therapy (OPT) applications. We implemented a fully automatic approach based on two-stage convolutional neural networks (CNNs): the first stage roughly identifies the eye position and the second one performs a fine iris and pupil detection. We selected 707 video frames recorded during clinical operations during OPT treatments performed at our institute. 650 frames were used for training and 57 for a blind test. The estimations of iris and pupil were evaluated against the manual labelled contours delineated by a clinical operator. For iris and pupil predictions, Dice coefficient (median = 0.94 and 0.97), Szymkiewicz–Simpson coefficient (median = 0.97 and 0.98), Intersection over Union coefficient (median = 0.88 and 0.94) and Hausdorff distance (median = 11.6 and 5.0 (pixels)) were quantified. Iris and pupil regions were found to be comparable to the manually labelled ground truths. Our proposed framework could provide an automatic approach to quantitatively evaluating pupil and iris misalignments, and it could be used as an additional support tool for clinical activity, without impacting in any way with the consolidated routine.https://www.mdpi.com/1424-8220/21/13/4400ocular proton therapyconvolutional neural networkseye trackingpupil segmentationiris segmentation |
| spellingShingle | Luca Antonioli Andrea Pella Rosalinda Ricotti Matteo Rossi Maria Rosaria Fiore Gabriele Belotti Giuseppe Magro Chiara Paganelli Ester Orlandi Mario Ciocca Guido Baroni Convolutional Neural Networks Cascade for Automatic Pupil and Iris Detection in Ocular Proton Therapy Sensors ocular proton therapy convolutional neural networks eye tracking pupil segmentation iris segmentation |
| title | Convolutional Neural Networks Cascade for Automatic Pupil and Iris Detection in Ocular Proton Therapy |
| title_full | Convolutional Neural Networks Cascade for Automatic Pupil and Iris Detection in Ocular Proton Therapy |
| title_fullStr | Convolutional Neural Networks Cascade for Automatic Pupil and Iris Detection in Ocular Proton Therapy |
| title_full_unstemmed | Convolutional Neural Networks Cascade for Automatic Pupil and Iris Detection in Ocular Proton Therapy |
| title_short | Convolutional Neural Networks Cascade for Automatic Pupil and Iris Detection in Ocular Proton Therapy |
| title_sort | convolutional neural networks cascade for automatic pupil and iris detection in ocular proton therapy |
| topic | ocular proton therapy convolutional neural networks eye tracking pupil segmentation iris segmentation |
| url | https://www.mdpi.com/1424-8220/21/13/4400 |
| work_keys_str_mv | AT lucaantonioli convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT andreapella convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT rosalindaricotti convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT matteorossi convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT mariarosariafiore convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT gabrielebelotti convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT giuseppemagro convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT chiarapaganelli convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT esterorlandi convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT mariociocca convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy AT guidobaroni convolutionalneuralnetworkscascadeforautomaticpupilandirisdetectioninocularprotontherapy |