Evaluating the Viability of a Smartphone-Based Annotation Tool for Faster and Accurate Image Labelling for Artificial Intelligence in Diabetic Retinopathy
Arvind Kumar Morya,1 Jaitra Gowdar,2 Abhishek Kaushal,2 Nachiket Makwana,2 Saurav Biswas,2 Puneeth Raj,2 Sharat Hegde,3 Girish Velis,4 Winston Padua,5 Nazneen Nazm,6 Sangeetha Jeganathan,7 Sandeep Choudhary,1 Nishant Parashar,1 Bhavana Sharma,8 Pankaja Raghav1 1Department of Ophthalmology, All India...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Dove Medical Press
2021-03-01
|
| Series: | Clinical Ophthalmology |
| Subjects: | |
| Online Access: | https://www.dovepress.com/evaluating-the-viability-of-a-smartphone-based-annotation-tool-for-fas-peer-reviewed-article-OPTH |
| _version_ | 1818425622269001728 |
|---|---|
| author | Morya AK Gowdar J Kaushal A Makwana N Biswas S Raj P Hegde S Velis G Padua W Nazm N Jeganathan S Choudhary S Parashar N Sharma B Raghav P |
| author_facet | Morya AK Gowdar J Kaushal A Makwana N Biswas S Raj P Hegde S Velis G Padua W Nazm N Jeganathan S Choudhary S Parashar N Sharma B Raghav P |
| author_sort | Morya AK |
| collection | DOAJ |
| description | Arvind Kumar Morya,1 Jaitra Gowdar,2 Abhishek Kaushal,2 Nachiket Makwana,2 Saurav Biswas,2 Puneeth Raj,2 Sharat Hegde,3 Girish Velis,4 Winston Padua,5 Nazneen Nazm,6 Sangeetha Jeganathan,7 Sandeep Choudhary,1 Nishant Parashar,1 Bhavana Sharma,8 Pankaja Raghav1 1Department of Ophthalmology, All India Institute of Medical Sciences, Jodhpur, Rajasthan, 342005, India; 2Radiate Healthcare Innovations Private Limited, Bangalore, Karnataka, 560038, India; 3Prasad Netralaya, Udupi, Karnataka, 576101, India; 4Goa Medical College, Bambolim, Goa, 403202, India; 5St.John’s Medical College & Hospital, Bengaluru, Karnataka, 560034, India; 6ESI PGIMSR, ESI Medical College and Hospital, Kolkata, West Bengal, 700104, India; 7Srinivas Institute of Medical Sciences and Research Centre, Mangalore, Karnataka, 574146, India; 8All India Institute of Medical Sciences, Bhopal, Madhya Pradesh, 462020, IndiaCorrespondence: Arvind Kumar MoryaDepartment of Ophthalmology, All India Institute of Medical Sciences, Bibinagar, Hyderabad, Telangana, 508126, IndiaEmail bulbul.morya@gmail.comIntroduction: Deep Learning (DL) and Artificial Intelligence (AI) have become widespread due to the advanced technologies and availability of digital data. Supervised learning algorithms have shown human-level performance or even better and are better feature extractor-quantifier than unsupervised learning algorithms. To get huge dataset with good quality control, there is a need of an annotation tool with a customizable feature set. This paper evaluates the viability of having an in house annotation tool which works on a smartphone and can be used in a healthcare setting.Methods: We developed a smartphone-based grading system to help researchers in grading multiple retinal fundi. The process consisted of designing the flow of user interface (UI) keeping in view feedback from experts. Quantitative and qualitative analysis of change in speed of a grader over time and feature usage statistics was done. The dataset size was approximately 16,000 images with adjudicated labels by a minimum of 2 doctors. Results for an AI model trained on the images graded using this tool and its validation over some public datasets were prepared.Results: We created a DL model and analysed its performance for a binary referrable DR Classification task, whether a retinal image has Referrable DR or not. A total of 32 doctors used the tool for minimum of 20 images each. Data analytics suggested significant portability and flexibility of the tool. Grader variability for images was in favour of agreement on images annotated. Number of images used to assess agreement is 550. Mean of 75.9% was seen in agreement.Conclusion: Our aim was to make Annotation of Medical imaging easier and to minimize time taken for annotations without quality degradation. The user feedback and feature usage statistics confirm our hypotheses of incorporation of brightness and contrast variations, green channels and zooming add-ons in correlation to certain disease types. Simulation of multiple review cycles and establishing quality control can boost the accuracy of AI models even further. Although our study aims at developing an annotation tool for diagnosing and classifying diabetic retinopathy fundus images but same concept can be used for fundus images of other ocular diseases as well as other streams of medical science such as radiology where image-based diagnostic applications are utilised.Keywords: artificial intelligence, deep learning, referrable diabetic retinopathy |
| first_indexed | 2024-12-14T14:16:52Z |
| format | Article |
| id | doaj.art-f61fbd143f6d419ab4e1a13078d399e4 |
| institution | Directory Open Access Journal |
| issn | 1177-5483 |
| language | English |
| last_indexed | 2024-12-14T14:16:52Z |
| publishDate | 2021-03-01 |
| publisher | Dove Medical Press |
| record_format | Article |
| series | Clinical Ophthalmology |
| spelling | doaj.art-f61fbd143f6d419ab4e1a13078d399e42022-12-21T22:58:11ZengDove Medical PressClinical Ophthalmology1177-54832021-03-01Volume 151023103962848Evaluating the Viability of a Smartphone-Based Annotation Tool for Faster and Accurate Image Labelling for Artificial Intelligence in Diabetic RetinopathyMorya AKGowdar JKaushal AMakwana NBiswas SRaj PHegde SVelis GPadua WNazm NJeganathan SChoudhary SParashar NSharma BRaghav PArvind Kumar Morya,1 Jaitra Gowdar,2 Abhishek Kaushal,2 Nachiket Makwana,2 Saurav Biswas,2 Puneeth Raj,2 Sharat Hegde,3 Girish Velis,4 Winston Padua,5 Nazneen Nazm,6 Sangeetha Jeganathan,7 Sandeep Choudhary,1 Nishant Parashar,1 Bhavana Sharma,8 Pankaja Raghav1 1Department of Ophthalmology, All India Institute of Medical Sciences, Jodhpur, Rajasthan, 342005, India; 2Radiate Healthcare Innovations Private Limited, Bangalore, Karnataka, 560038, India; 3Prasad Netralaya, Udupi, Karnataka, 576101, India; 4Goa Medical College, Bambolim, Goa, 403202, India; 5St.John’s Medical College & Hospital, Bengaluru, Karnataka, 560034, India; 6ESI PGIMSR, ESI Medical College and Hospital, Kolkata, West Bengal, 700104, India; 7Srinivas Institute of Medical Sciences and Research Centre, Mangalore, Karnataka, 574146, India; 8All India Institute of Medical Sciences, Bhopal, Madhya Pradesh, 462020, IndiaCorrespondence: Arvind Kumar MoryaDepartment of Ophthalmology, All India Institute of Medical Sciences, Bibinagar, Hyderabad, Telangana, 508126, IndiaEmail bulbul.morya@gmail.comIntroduction: Deep Learning (DL) and Artificial Intelligence (AI) have become widespread due to the advanced technologies and availability of digital data. Supervised learning algorithms have shown human-level performance or even better and are better feature extractor-quantifier than unsupervised learning algorithms. To get huge dataset with good quality control, there is a need of an annotation tool with a customizable feature set. This paper evaluates the viability of having an in house annotation tool which works on a smartphone and can be used in a healthcare setting.Methods: We developed a smartphone-based grading system to help researchers in grading multiple retinal fundi. The process consisted of designing the flow of user interface (UI) keeping in view feedback from experts. Quantitative and qualitative analysis of change in speed of a grader over time and feature usage statistics was done. The dataset size was approximately 16,000 images with adjudicated labels by a minimum of 2 doctors. Results for an AI model trained on the images graded using this tool and its validation over some public datasets were prepared.Results: We created a DL model and analysed its performance for a binary referrable DR Classification task, whether a retinal image has Referrable DR or not. A total of 32 doctors used the tool for minimum of 20 images each. Data analytics suggested significant portability and flexibility of the tool. Grader variability for images was in favour of agreement on images annotated. Number of images used to assess agreement is 550. Mean of 75.9% was seen in agreement.Conclusion: Our aim was to make Annotation of Medical imaging easier and to minimize time taken for annotations without quality degradation. The user feedback and feature usage statistics confirm our hypotheses of incorporation of brightness and contrast variations, green channels and zooming add-ons in correlation to certain disease types. Simulation of multiple review cycles and establishing quality control can boost the accuracy of AI models even further. Although our study aims at developing an annotation tool for diagnosing and classifying diabetic retinopathy fundus images but same concept can be used for fundus images of other ocular diseases as well as other streams of medical science such as radiology where image-based diagnostic applications are utilised.Keywords: artificial intelligence, deep learning, referrable diabetic retinopathyhttps://www.dovepress.com/evaluating-the-viability-of-a-smartphone-based-annotation-tool-for-fas-peer-reviewed-article-OPTHartificial intelligencedeep learningreferrable diabetic retinopathy |
| spellingShingle | Morya AK Gowdar J Kaushal A Makwana N Biswas S Raj P Hegde S Velis G Padua W Nazm N Jeganathan S Choudhary S Parashar N Sharma B Raghav P Evaluating the Viability of a Smartphone-Based Annotation Tool for Faster and Accurate Image Labelling for Artificial Intelligence in Diabetic Retinopathy Clinical Ophthalmology artificial intelligence deep learning referrable diabetic retinopathy |
| title | Evaluating the Viability of a Smartphone-Based Annotation Tool for Faster and Accurate Image Labelling for Artificial Intelligence in Diabetic Retinopathy |
| title_full | Evaluating the Viability of a Smartphone-Based Annotation Tool for Faster and Accurate Image Labelling for Artificial Intelligence in Diabetic Retinopathy |
| title_fullStr | Evaluating the Viability of a Smartphone-Based Annotation Tool for Faster and Accurate Image Labelling for Artificial Intelligence in Diabetic Retinopathy |
| title_full_unstemmed | Evaluating the Viability of a Smartphone-Based Annotation Tool for Faster and Accurate Image Labelling for Artificial Intelligence in Diabetic Retinopathy |
| title_short | Evaluating the Viability of a Smartphone-Based Annotation Tool for Faster and Accurate Image Labelling for Artificial Intelligence in Diabetic Retinopathy |
| title_sort | evaluating the viability of a smartphone based annotation tool for faster and accurate image labelling for artificial intelligence in diabetic retinopathy |
| topic | artificial intelligence deep learning referrable diabetic retinopathy |
| url | https://www.dovepress.com/evaluating-the-viability-of-a-smartphone-based-annotation-tool-for-fas-peer-reviewed-article-OPTH |
| work_keys_str_mv | AT moryaak evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT gowdarj evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT kaushala evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT makwanan evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT biswass evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT rajp evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT hegdes evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT velisg evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT paduaw evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT nazmn evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT jeganathans evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT choudharys evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT parasharn evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT sharmab evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy AT raghavp evaluatingtheviabilityofasmartphonebasedannotationtoolforfasterandaccurateimagelabellingforartificialintelligenceindiabeticretinopathy |