Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning
Abstract Background Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantif...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-04-01
|
| Series: | Malaria Journal |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12936-022-04146-1 |
| _version_ | 1811332530127765504 |
|---|---|
| author | Debashish Das Ranitha Vongpromek Thanawat Assawariyathipat Ketsanee Srinamon Kalynn Kennon Kasia Stepniewska Aniruddha Ghose Abdullah Abu Sayeed M. Abul Faiz Rebeca Linhares Abreu Netto Andre Siqueira Serge R. Yerbanga Jean Bosco Ouédraogo James J. Callery Thomas J. Peto Rupam Tripura Felix Koukouikila-Koussounda Francine Ntoumi John Michael Ong’echa Bernhards Ogutu Prakash Ghimire Jutta Marfurt Benedikt Ley Amadou Seck Magatte Ndiaye Bhavani Moodley Lisa Ming Sun Laypaw Archasuksan Stephane Proux Sam L. Nsobya Philip J. Rosenthal Matthew P. Horning Shawn K. McGuire Courosh Mehanian Stephen Burkot Charles B. Delahunt Christine Bachman Ric N. Price Arjen M. Dondorp François Chappuis Philippe J. Guérin Mehul Dhorda |
| author_facet | Debashish Das Ranitha Vongpromek Thanawat Assawariyathipat Ketsanee Srinamon Kalynn Kennon Kasia Stepniewska Aniruddha Ghose Abdullah Abu Sayeed M. Abul Faiz Rebeca Linhares Abreu Netto Andre Siqueira Serge R. Yerbanga Jean Bosco Ouédraogo James J. Callery Thomas J. Peto Rupam Tripura Felix Koukouikila-Koussounda Francine Ntoumi John Michael Ong’echa Bernhards Ogutu Prakash Ghimire Jutta Marfurt Benedikt Ley Amadou Seck Magatte Ndiaye Bhavani Moodley Lisa Ming Sun Laypaw Archasuksan Stephane Proux Sam L. Nsobya Philip J. Rosenthal Matthew P. Horning Shawn K. McGuire Courosh Mehanian Stephen Burkot Charles B. Delahunt Christine Bachman Ric N. Price Arjen M. Dondorp François Chappuis Philippe J. Guérin Mehul Dhorda |
| author_sort | Debashish Das |
| collection | DOAJ |
| description | Abstract Background Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue. Methods A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. Results In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9–92.7), and specificity 75.6% (95% CI 73.1–78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2–91.5), but specificity increased to 85.1% (95%CI 82.6–87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200–200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69–0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66–0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides. Conclusions The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678. |
| first_indexed | 2024-04-13T16:38:19Z |
| format | Article |
| id | doaj.art-9f3429c323c04ff7bc896f60dfbe9446 |
| institution | Directory Open Access Journal |
| issn | 1475-2875 |
| language | English |
| last_indexed | 2024-04-13T16:38:19Z |
| publishDate | 2022-04-01 |
| publisher | BMC |
| record_format | Article |
| series | Malaria Journal |
| spelling | doaj.art-9f3429c323c04ff7bc896f60dfbe94462022-12-22T02:39:22ZengBMCMalaria Journal1475-28752022-04-0121111210.1186/s12936-022-04146-1Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learningDebashish Das0Ranitha Vongpromek1Thanawat Assawariyathipat2Ketsanee Srinamon3Kalynn Kennon4Kasia Stepniewska5Aniruddha Ghose6Abdullah Abu Sayeed7M. Abul Faiz8Rebeca Linhares Abreu Netto9Andre Siqueira10Serge R. Yerbanga11Jean Bosco Ouédraogo12James J. Callery13Thomas J. Peto14Rupam Tripura15Felix Koukouikila-Koussounda16Francine Ntoumi17John Michael Ong’echa18Bernhards Ogutu19Prakash Ghimire20Jutta Marfurt21Benedikt Ley22Amadou Seck23Magatte Ndiaye24Bhavani Moodley25Lisa Ming Sun26Laypaw Archasuksan27Stephane Proux28Sam L. Nsobya29Philip J. Rosenthal30Matthew P. Horning31Shawn K. McGuire32Courosh Mehanian33Stephen Burkot34Charles B. Delahunt35Christine Bachman36Ric N. Price37Arjen M. Dondorp38François Chappuis39Philippe J. Guérin40Mehul Dhorda41Infectious Diseases Data Observatory (IDDO)Infectious Diseases Data Observatory (IDDO)Infectious Diseases Data Observatory (IDDO)Faculty of Tropical Medicine, Mahidol-Oxford Tropical Medicine Research Unit, Mahidol UniversityInfectious Diseases Data Observatory (IDDO)Infectious Diseases Data Observatory (IDDO)Chittagong Medical College (CMC)Chittagong Medical College (CMC)Dev Care FoundationFundação de Medicina Tropical Dr Heitor Vieira DouradoOswaldo Cruz Foundation (Fiocruz)Institut Des Sciences Et Techniques (INSTech)Institut Des Sciences Et Techniques (INSTech)Centre for Tropical Medicine and Global Health, Nuffield Department of Medicine, University of OxfordCentre for Tropical Medicine and Global Health, Nuffield Department of Medicine, University of OxfordCentre for Tropical Medicine and Global Health, Nuffield Department of Medicine, University of OxfordFondation Congolaise Pour La Recherche Médicale (FCRM)Fondation Congolaise Pour La Recherche Médicale (FCRM)Kenya Medical Research Institute (KEMRI)Kenya Medical Research Institute (KEMRI)Tribhuvan UniversityGlobal and Tropical Health Division, Menzies School of Health Research, Charles Darwin UniversityGlobal and Tropical Health Division, Menzies School of Health Research, Charles Darwin UniversityFaculty of Medicine, University Cheikh Anta Diop (UCAD)Faculty of Medicine, University Cheikh Anta Diop (UCAD)Parasitology Reference Laboratory, National Institute for Communicable Diseases, Division of the National Health Laboratory ServiceParasitology Reference Laboratory, National Institute for Communicable Diseases, Division of the National Health Laboratory ServiceShoklo Malaria Research Unit, Mahidol-Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine, Mahidol UniversityShoklo Malaria Research Unit, Mahidol-Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine, Mahidol UniversityDepartment of Pathology, College of Health Science, Makerere UniversityUniversity of CaliforniaGlobal Health LabsGlobal Health LabsGlobal Health LabsGlobal Health LabsGlobal Health LabsGlobal Health LabsCentre for Tropical Medicine and Global Health, Nuffield Department of Medicine, University of OxfordCentre for Tropical Medicine and Global Health, Nuffield Department of Medicine, University of OxfordDivision of Tropical and Humanitarian Medicine, Geneva University Hospitals and University of GenevaInfectious Diseases Data Observatory (IDDO)Infectious Diseases Data Observatory (IDDO)Abstract Background Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue. Methods A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. Results In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9–92.7), and specificity 75.6% (95% CI 73.1–78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2–91.5), but specificity increased to 85.1% (95%CI 82.6–87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200–200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69–0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66–0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides. Conclusions The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678.https://doi.org/10.1186/s12936-022-04146-1MalariaLight microscopyDigital microscopyArtificial intelligenceDiagnostic accuracyMachine-learning |
| spellingShingle | Debashish Das Ranitha Vongpromek Thanawat Assawariyathipat Ketsanee Srinamon Kalynn Kennon Kasia Stepniewska Aniruddha Ghose Abdullah Abu Sayeed M. Abul Faiz Rebeca Linhares Abreu Netto Andre Siqueira Serge R. Yerbanga Jean Bosco Ouédraogo James J. Callery Thomas J. Peto Rupam Tripura Felix Koukouikila-Koussounda Francine Ntoumi John Michael Ong’echa Bernhards Ogutu Prakash Ghimire Jutta Marfurt Benedikt Ley Amadou Seck Magatte Ndiaye Bhavani Moodley Lisa Ming Sun Laypaw Archasuksan Stephane Proux Sam L. Nsobya Philip J. Rosenthal Matthew P. Horning Shawn K. McGuire Courosh Mehanian Stephen Burkot Charles B. Delahunt Christine Bachman Ric N. Price Arjen M. Dondorp François Chappuis Philippe J. Guérin Mehul Dhorda Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning Malaria Journal Malaria Light microscopy Digital microscopy Artificial intelligence Diagnostic accuracy Machine-learning |
| title | Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning |
| title_full | Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning |
| title_fullStr | Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning |
| title_full_unstemmed | Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning |
| title_short | Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning |
| title_sort | field evaluation of the diagnostic performance of easyscan go a digital malaria microscopy device based on machine learning |
| topic | Malaria Light microscopy Digital microscopy Artificial intelligence Diagnostic accuracy Machine-learning |
| url | https://doi.org/10.1186/s12936-022-04146-1 |
| work_keys_str_mv | AT debashishdas fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT ranithavongpromek fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT thanawatassawariyathipat fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT ketsaneesrinamon fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT kalynnkennon fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT kasiastepniewska fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT aniruddhaghose fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT abdullahabusayeed fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT mabulfaiz fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT rebecalinharesabreunetto fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT andresiqueira fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT sergeryerbanga fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT jeanboscoouedraogo fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT jamesjcallery fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT thomasjpeto fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT rupamtripura fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT felixkoukouikilakoussounda fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT francinentoumi fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT johnmichaelongecha fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT bernhardsogutu fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT prakashghimire fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT juttamarfurt fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT benediktley fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT amadouseck fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT magattendiaye fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT bhavanimoodley fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT lisamingsun fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT laypawarchasuksan fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT stephaneproux fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT samlnsobya fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT philipjrosenthal fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT matthewphorning fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT shawnkmcguire fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT couroshmehanian fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT stephenburkot fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT charlesbdelahunt fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT christinebachman fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT ricnprice fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT arjenmdondorp fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT francoischappuis fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT philippejguerin fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning AT mehuldhorda fieldevaluationofthediagnosticperformanceofeasyscangoadigitalmalariamicroscopydevicebasedonmachinelearning |