Patient-level performance evaluation of a smartphone-based malaria diagnostic application
Abstract Background Microscopic examination is commonly used for malaria diagnosis in the field. However, the lack of well-trained microscopists in malaria-endemic areas impacted the most by the disease is a severe problem. Besides, the examination process is time-consuming and prone to human error....
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-01-01
|
| Series: | Malaria Journal |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12936-023-04446-0 |
| _version_ | 1811176028936077312 |
|---|---|
| author | Hang Yu Fayad O. Mohammed Muzamil Abdel Hamid Feng Yang Yasmin M. Kassim Abdelrahim O. Mohamed Richard J. Maude Xavier C. Ding Ewurama D.A. Owusu Seda Yerlikaya Sabine Dittrich Stefan Jaeger |
| author_facet | Hang Yu Fayad O. Mohammed Muzamil Abdel Hamid Feng Yang Yasmin M. Kassim Abdelrahim O. Mohamed Richard J. Maude Xavier C. Ding Ewurama D.A. Owusu Seda Yerlikaya Sabine Dittrich Stefan Jaeger |
| author_sort | Hang Yu |
| collection | DOAJ |
| description | Abstract Background Microscopic examination is commonly used for malaria diagnosis in the field. However, the lack of well-trained microscopists in malaria-endemic areas impacted the most by the disease is a severe problem. Besides, the examination process is time-consuming and prone to human error. Automated diagnostic systems based on machine learning offer great potential to overcome these problems. This study aims to evaluate Malaria Screener, a smartphone-based application for malaria diagnosis. Methods A total of 190 patients were recruited at two sites in rural areas near Khartoum, Sudan. The Malaria Screener mobile application was deployed to screen Giemsa-stained blood smears. Both expert microscopy and nested PCR were performed to use as reference standards. First, Malaria Screener was evaluated using the two reference standards. Then, during post-study experiments, the evaluation was repeated for a newly developed algorithm, PlasmodiumVF-Net. Results Malaria Screener reached 74.1% (95% CI 63.5–83.0) accuracy in detecting Plasmodium falciparum malaria using expert microscopy as the reference after a threshold calibration. It reached 71.8% (95% CI 61.0–81.0) accuracy when compared with PCR. The achieved accuracies meet the WHO Level 3 requirement for parasite detection. The processing time for each smear varies from 5 to 15 min, depending on the concentration of white blood cells (WBCs). In the post-study experiment, Malaria Screener reached 91.8% (95% CI 83.8–96.6) accuracy when patient-level results were calculated with a different method. This accuracy meets the WHO Level 1 requirement for parasite detection. In addition, PlasmodiumVF-Net, a newly developed algorithm, reached 83.1% (95% CI 77.0–88.1) accuracy when compared with expert microscopy and 81.0% (95% CI 74.6–86.3) accuracy when compared with PCR, reaching the WHO Level 2 requirement for detecting both Plasmodium falciparum and Plasmodium vivax malaria, without using the testing sites data for training or calibration. Results reported for both Malaria Screener and PlasmodiumVF-Net used thick smears for diagnosis. In this paper, both systems were not assessed in species identification and parasite counting, which are still under development. Conclusion Malaria Screener showed the potential to be deployed in resource-limited areas to facilitate routine malaria screening. It is the first smartphone-based system for malaria diagnosis evaluated on the patient-level in a natural field environment. Thus, the results in the field reported here can serve as a reference for future studies. |
| first_indexed | 2024-04-10T19:45:21Z |
| format | Article |
| id | doaj.art-8a01579177264c2fa6a9617e6c1e9a26 |
| institution | Directory Open Access Journal |
| issn | 1475-2875 |
| language | English |
| last_indexed | 2024-04-10T19:45:21Z |
| publishDate | 2023-01-01 |
| publisher | BMC |
| record_format | Article |
| series | Malaria Journal |
| spelling | doaj.art-8a01579177264c2fa6a9617e6c1e9a262023-01-29T12:04:54ZengBMCMalaria Journal1475-28752023-01-0122111010.1186/s12936-023-04446-0Patient-level performance evaluation of a smartphone-based malaria diagnostic applicationHang Yu0Fayad O. Mohammed1Muzamil Abdel Hamid2Feng Yang3Yasmin M. Kassim4Abdelrahim O. Mohamed5Richard J. Maude6Xavier C. Ding7Ewurama D.A. Owusu8Seda Yerlikaya9Sabine Dittrich10Stefan Jaeger11Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of HealthDepartment of Parasitology and Medical Entomology, Institute of Endemic Diseases, Medical Campus, University of KhartoumDepartment of Parasitology and Medical Entomology, Institute of Endemic Diseases, Medical Campus, University of KhartoumLister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of HealthLister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of HealthDepartment of Parasitology and Medical Entomology, Institute of Endemic Diseases, Medical Campus, University of KhartoumMahidol Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine, Mahidol UniversityFINDFINDFINDFINDLister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of HealthAbstract Background Microscopic examination is commonly used for malaria diagnosis in the field. However, the lack of well-trained microscopists in malaria-endemic areas impacted the most by the disease is a severe problem. Besides, the examination process is time-consuming and prone to human error. Automated diagnostic systems based on machine learning offer great potential to overcome these problems. This study aims to evaluate Malaria Screener, a smartphone-based application for malaria diagnosis. Methods A total of 190 patients were recruited at two sites in rural areas near Khartoum, Sudan. The Malaria Screener mobile application was deployed to screen Giemsa-stained blood smears. Both expert microscopy and nested PCR were performed to use as reference standards. First, Malaria Screener was evaluated using the two reference standards. Then, during post-study experiments, the evaluation was repeated for a newly developed algorithm, PlasmodiumVF-Net. Results Malaria Screener reached 74.1% (95% CI 63.5–83.0) accuracy in detecting Plasmodium falciparum malaria using expert microscopy as the reference after a threshold calibration. It reached 71.8% (95% CI 61.0–81.0) accuracy when compared with PCR. The achieved accuracies meet the WHO Level 3 requirement for parasite detection. The processing time for each smear varies from 5 to 15 min, depending on the concentration of white blood cells (WBCs). In the post-study experiment, Malaria Screener reached 91.8% (95% CI 83.8–96.6) accuracy when patient-level results were calculated with a different method. This accuracy meets the WHO Level 1 requirement for parasite detection. In addition, PlasmodiumVF-Net, a newly developed algorithm, reached 83.1% (95% CI 77.0–88.1) accuracy when compared with expert microscopy and 81.0% (95% CI 74.6–86.3) accuracy when compared with PCR, reaching the WHO Level 2 requirement for detecting both Plasmodium falciparum and Plasmodium vivax malaria, without using the testing sites data for training or calibration. Results reported for both Malaria Screener and PlasmodiumVF-Net used thick smears for diagnosis. In this paper, both systems were not assessed in species identification and parasite counting, which are still under development. Conclusion Malaria Screener showed the potential to be deployed in resource-limited areas to facilitate routine malaria screening. It is the first smartphone-based system for malaria diagnosis evaluated on the patient-level in a natural field environment. Thus, the results in the field reported here can serve as a reference for future studies.https://doi.org/10.1186/s12936-023-04446-0Malaria microscopyComputer-aided diagnosisAutomated screeningMachine learningField testingSmartphone application |
| spellingShingle | Hang Yu Fayad O. Mohammed Muzamil Abdel Hamid Feng Yang Yasmin M. Kassim Abdelrahim O. Mohamed Richard J. Maude Xavier C. Ding Ewurama D.A. Owusu Seda Yerlikaya Sabine Dittrich Stefan Jaeger Patient-level performance evaluation of a smartphone-based malaria diagnostic application Malaria Journal Malaria microscopy Computer-aided diagnosis Automated screening Machine learning Field testing Smartphone application |
| title | Patient-level performance evaluation of a smartphone-based malaria diagnostic application |
| title_full | Patient-level performance evaluation of a smartphone-based malaria diagnostic application |
| title_fullStr | Patient-level performance evaluation of a smartphone-based malaria diagnostic application |
| title_full_unstemmed | Patient-level performance evaluation of a smartphone-based malaria diagnostic application |
| title_short | Patient-level performance evaluation of a smartphone-based malaria diagnostic application |
| title_sort | patient level performance evaluation of a smartphone based malaria diagnostic application |
| topic | Malaria microscopy Computer-aided diagnosis Automated screening Machine learning Field testing Smartphone application |
| url | https://doi.org/10.1186/s12936-023-04446-0 |
| work_keys_str_mv | AT hangyu patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT fayadomohammed patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT muzamilabdelhamid patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT fengyang patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT yasminmkassim patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT abdelrahimomohamed patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT richardjmaude patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT xaviercding patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT ewuramadaowusu patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT sedayerlikaya patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT sabinedittrich patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication AT stefanjaeger patientlevelperformanceevaluationofasmartphonebasedmalariadiagnosticapplication |