Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care
Abstract Dermatological conditions are a relevant health problem. Machine learning (ML) models are increasingly being applied to dermatology as a diagnostic decision support tool using image analysis, especially for skin cancer detection and disease classification. The objective of this study was to...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-03-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-31340-1 |
| _version_ | 1797864804755439616 |
|---|---|
| author | Anna Escalé-Besa Oriol Yélamos Josep Vidal-Alaball Aïna Fuster-Casanovas Queralt Miró Catalina Alexander Börve Ricardo Ander-Egg Aguilar Xavier Fustà-Novell Xavier Cubiró Mireia Esquius Rafat Cristina López-Sanchez Francesc X. Marin-Gomez |
| author_facet | Anna Escalé-Besa Oriol Yélamos Josep Vidal-Alaball Aïna Fuster-Casanovas Queralt Miró Catalina Alexander Börve Ricardo Ander-Egg Aguilar Xavier Fustà-Novell Xavier Cubiró Mireia Esquius Rafat Cristina López-Sanchez Francesc X. Marin-Gomez |
| author_sort | Anna Escalé-Besa |
| collection | DOAJ |
| description | Abstract Dermatological conditions are a relevant health problem. Machine learning (ML) models are increasingly being applied to dermatology as a diagnostic decision support tool using image analysis, especially for skin cancer detection and disease classification. The objective of this study was to perform a prospective validation of an image analysis ML model, which is capable of screening 44 skin diseases, comparing its diagnostic accuracy with that of General Practitioners (GPs) and teledermatology (TD) dermatologists in a real-life setting. Prospective, diagnostic accuracy study including 100 consecutive patients with a skin problem who visited a participating GP in central Catalonia, Spain, between June 2021 and October 2021. The skin issue was first assessed by the GPs. Then an anonymised skin disease picture was taken and uploaded to the ML application, which returned a list with the Top-5 possible diagnosis in order of probability. The same image was then sent to a dermatologist via TD for diagnosis, as per clinical practice. The GPs Top-3, ML model’s Top-5 and dermatologist’s Top-3 assessments were compared to calculate the accuracy, sensitivity, specificity and diagnostic accuracy of the ML models. The overall Top-1 accuracy of the ML model (39%) was lower than that of GPs (64%) and dermatologists (72%). When the analysis was limited to the diagnoses on which the algorithm had been explicitly trained (n = 82), the balanced Top-1 accuracy of the ML model increased (48%) and in the Top-3 (75%) was comparable to the GPs Top-3 accuracy (76%). The Top-5 accuracy of the ML model (89%) was comparable to the dermatologist Top-3 accuracy (90%). For the different diseases, the sensitivity of the model (Top-3 87% and Top-5 96%) is higher than that of the clinicians (Top-3 GPs 76% and Top-3 dermatologists 84%) only in the benign tumour pathology group, being on the other hand the most prevalent category (n = 53). About the satisfaction of professionals, 92% of the GPs considered it as a useful diagnostic support tool (DST) for the differential diagnosis and in 60% of the cases as an aid in the final diagnosis of the skin lesion. The overall diagnostic accuracy of the model in this study, under real-life conditions, is lower than that of both GPs and dermatologists. This result aligns with the findings of few existing prospective studies conducted under real-life conditions. The outcomes emphasize the significance of involving clinicians in the training of the model and the capability of ML models to assist GPs, particularly in differential diagnosis. Nevertheless, external testing in real-life conditions is crucial for data validation and regulation of these AI diagnostic models before they can be used in primary care. |
| first_indexed | 2024-04-09T22:59:02Z |
| format | Article |
| id | doaj.art-0d5187ed334546d2aca34e9f870f9dc6 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-09T22:59:02Z |
| publishDate | 2023-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-0d5187ed334546d2aca34e9f870f9dc62023-03-22T11:05:05ZengNature PortfolioScientific Reports2045-23222023-03-0113111410.1038/s41598-023-31340-1Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary careAnna Escalé-Besa0Oriol Yélamos1Josep Vidal-Alaball2Aïna Fuster-Casanovas3Queralt Miró Catalina4Alexander Börve5Ricardo Ander-Egg Aguilar6Xavier Fustà-Novell7Xavier Cubiró8Mireia Esquius Rafat9Cristina López-Sanchez10Francesc X. Marin-Gomez11Centre d’Atenció Primària Navàs-Balsareny, Institut Català de la SalutDermatology Department, Hospital de la Santa Creu i Sant Pau, Universitat Autònoma de BarcelonaHealth Promotion in Rural Areas Research Group, Gerència Territorial de la Catalunya Central, Institut Català de la SalutHealth Promotion in Rural Areas Research Group, Gerència Territorial de la Catalunya Central, Institut Català de la SalutHealth Promotion in Rural Areas Research Group, Gerència Territorial de la Catalunya Central, Institut Català de la SalutiDoc24 InciDoc24 IncFundació Althaia de ManresaServei de Dermatologia, Hospital Universitari MolletFundació Althaia de ManresaDermatology Department, Hospital de la Santa Creu i Sant Pau, Universitat Autònoma de BarcelonaHealth Promotion in Rural Areas Research Group, Gerència Territorial de la Catalunya Central, Institut Català de la SalutAbstract Dermatological conditions are a relevant health problem. Machine learning (ML) models are increasingly being applied to dermatology as a diagnostic decision support tool using image analysis, especially for skin cancer detection and disease classification. The objective of this study was to perform a prospective validation of an image analysis ML model, which is capable of screening 44 skin diseases, comparing its diagnostic accuracy with that of General Practitioners (GPs) and teledermatology (TD) dermatologists in a real-life setting. Prospective, diagnostic accuracy study including 100 consecutive patients with a skin problem who visited a participating GP in central Catalonia, Spain, between June 2021 and October 2021. The skin issue was first assessed by the GPs. Then an anonymised skin disease picture was taken and uploaded to the ML application, which returned a list with the Top-5 possible diagnosis in order of probability. The same image was then sent to a dermatologist via TD for diagnosis, as per clinical practice. The GPs Top-3, ML model’s Top-5 and dermatologist’s Top-3 assessments were compared to calculate the accuracy, sensitivity, specificity and diagnostic accuracy of the ML models. The overall Top-1 accuracy of the ML model (39%) was lower than that of GPs (64%) and dermatologists (72%). When the analysis was limited to the diagnoses on which the algorithm had been explicitly trained (n = 82), the balanced Top-1 accuracy of the ML model increased (48%) and in the Top-3 (75%) was comparable to the GPs Top-3 accuracy (76%). The Top-5 accuracy of the ML model (89%) was comparable to the dermatologist Top-3 accuracy (90%). For the different diseases, the sensitivity of the model (Top-3 87% and Top-5 96%) is higher than that of the clinicians (Top-3 GPs 76% and Top-3 dermatologists 84%) only in the benign tumour pathology group, being on the other hand the most prevalent category (n = 53). About the satisfaction of professionals, 92% of the GPs considered it as a useful diagnostic support tool (DST) for the differential diagnosis and in 60% of the cases as an aid in the final diagnosis of the skin lesion. The overall diagnostic accuracy of the model in this study, under real-life conditions, is lower than that of both GPs and dermatologists. This result aligns with the findings of few existing prospective studies conducted under real-life conditions. The outcomes emphasize the significance of involving clinicians in the training of the model and the capability of ML models to assist GPs, particularly in differential diagnosis. Nevertheless, external testing in real-life conditions is crucial for data validation and regulation of these AI diagnostic models before they can be used in primary care.https://doi.org/10.1038/s41598-023-31340-1 |
| spellingShingle | Anna Escalé-Besa Oriol Yélamos Josep Vidal-Alaball Aïna Fuster-Casanovas Queralt Miró Catalina Alexander Börve Ricardo Ander-Egg Aguilar Xavier Fustà-Novell Xavier Cubiró Mireia Esquius Rafat Cristina López-Sanchez Francesc X. Marin-Gomez Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care Scientific Reports |
| title | Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care |
| title_full | Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care |
| title_fullStr | Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care |
| title_full_unstemmed | Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care |
| title_short | Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care |
| title_sort | exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care |
| url | https://doi.org/10.1038/s41598-023-31340-1 |
| work_keys_str_mv | AT annaescalebesa exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT oriolyelamos exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT josepvidalalaball exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT ainafustercasanovas exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT queraltmirocatalina exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT alexanderborve exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT ricardoandereggaguilar exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT xavierfustanovell exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT xaviercubiro exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT mireiaesquiusrafat exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT cristinalopezsanchez exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare AT francescxmaringomez exploringthepotentialofartificialintelligenceinimprovingskinlesiondiagnosisinprimarycare |