Algorithm-assisted diagnosis of Hirschsprung’s disease – evaluation of robustness and comparative image analysis on data from various labs and slide scanners
Abstract Background Differences in the preparation, staining and scanning of digital pathology slides create significant pre-analytic variability. Algorithm-assisted tools must be able to contend with this variability in order to be applicable in clinical practice. In a previous study, a decision su...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-02-01
|
| Series: | Diagnostic Pathology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13000-024-01452-x |
| _version_ | 1797276073781624832 |
|---|---|
| author | Ariel Greenberg Benzion Samueli Shai Farkash Yaniv Zohar Shahar Ish-Shalom Rami R. Hagege Dov Hershkovitz |
| author_facet | Ariel Greenberg Benzion Samueli Shai Farkash Yaniv Zohar Shahar Ish-Shalom Rami R. Hagege Dov Hershkovitz |
| author_sort | Ariel Greenberg |
| collection | DOAJ |
| description | Abstract Background Differences in the preparation, staining and scanning of digital pathology slides create significant pre-analytic variability. Algorithm-assisted tools must be able to contend with this variability in order to be applicable in clinical practice. In a previous study, a decision support algorithm was developed to assist in the diagnosis of Hirschsprung's disease. In the current study, we tested the robustness of this algorithm while assessing for pre-analytic factors which may affect its performance. Methods The decision support algorithm was used on digital pathology slides obtained from four different medical centers (A-D) and scanned by three different scanner models (by Philips, Hamamatsu and 3DHISTECH). A total of 192 cases and 1782 slides were used in this study. RGB histograms were constructed to compare images from the various medical centers and scanner models and highlight the differences in color and contrast. Results The algorithm was able to correctly identify ganglion cells in 99.2% of cases, from all medical centers (All scanned by the Philips slide scanner) as well as 95.5% and 100% of the slides scanned by the 3DHISTECH and Hamamatsu brand slide scanners, respectively. The total error rate for center D was lower than the other medical centers (3.9% vs 7.1%, 10.8% and 6% for centers A-C, respectively), the vast majority of errors being false positives (3.45% vs 0.45% false negatives). The other medical centers showed a higher rate of false negatives in relation to false positives (6.81% vs 0.29%, 9.8% vs 1.2% and 5.37% vs 0.63% for centers A-C, respectively). The total error rates for the Philips, Hamamatsu and 3DHISTECH brand scanners were 3.9%, 3.2% and 9.8%, respectively. RGB histograms demonstrated significant differences in pixel value distribution between the four medical centers, as well as between the 3DHISTECH brand scanner when compared to the Philips and Hamamatsu brand scanners. Conclusions The results reported in this paper suggest that the algorithm-based decision support system has sufficient robustness to be applicable for clinical practice. In addition, the novel method used in its development – Hierarchial-Contexual Analysis (HCA) may be applicable to the development of algorithm-assisted tools in other diseases, for which available datasets are limited. Validation of any given algorithm-assisted support system should nonetheless include data from as many medical centers and scanner models as possible. |
| first_indexed | 2024-03-07T15:23:06Z |
| format | Article |
| id | doaj.art-9de80ebf886c4e5da20392a77bc388de |
| institution | Directory Open Access Journal |
| issn | 1746-1596 |
| language | English |
| last_indexed | 2024-03-07T15:23:06Z |
| publishDate | 2024-02-01 |
| publisher | BMC |
| record_format | Article |
| series | Diagnostic Pathology |
| spelling | doaj.art-9de80ebf886c4e5da20392a77bc388de2024-03-05T17:30:24ZengBMCDiagnostic Pathology1746-15962024-02-0119111510.1186/s13000-024-01452-xAlgorithm-assisted diagnosis of Hirschsprung’s disease – evaluation of robustness and comparative image analysis on data from various labs and slide scannersAriel Greenberg0Benzion Samueli1Shai Farkash2Yaniv Zohar3Shahar Ish-Shalom4Rami R. Hagege5Dov Hershkovitz6Institute of Pathology, Tel-Aviv Sourasky Medical CenterDepartment of Pathology, Soroka University Medical CenterDepartment of Pathology, Emek Medical CenterDepartment of Pathology, Rambam Medical CenterDepartment of Pathology, Kaplan Medical CenterInstitute of Pathology, Tel-Aviv Sourasky Medical CenterInstitute of Pathology, Tel-Aviv Sourasky Medical CenterAbstract Background Differences in the preparation, staining and scanning of digital pathology slides create significant pre-analytic variability. Algorithm-assisted tools must be able to contend with this variability in order to be applicable in clinical practice. In a previous study, a decision support algorithm was developed to assist in the diagnosis of Hirschsprung's disease. In the current study, we tested the robustness of this algorithm while assessing for pre-analytic factors which may affect its performance. Methods The decision support algorithm was used on digital pathology slides obtained from four different medical centers (A-D) and scanned by three different scanner models (by Philips, Hamamatsu and 3DHISTECH). A total of 192 cases and 1782 slides were used in this study. RGB histograms were constructed to compare images from the various medical centers and scanner models and highlight the differences in color and contrast. Results The algorithm was able to correctly identify ganglion cells in 99.2% of cases, from all medical centers (All scanned by the Philips slide scanner) as well as 95.5% and 100% of the slides scanned by the 3DHISTECH and Hamamatsu brand slide scanners, respectively. The total error rate for center D was lower than the other medical centers (3.9% vs 7.1%, 10.8% and 6% for centers A-C, respectively), the vast majority of errors being false positives (3.45% vs 0.45% false negatives). The other medical centers showed a higher rate of false negatives in relation to false positives (6.81% vs 0.29%, 9.8% vs 1.2% and 5.37% vs 0.63% for centers A-C, respectively). The total error rates for the Philips, Hamamatsu and 3DHISTECH brand scanners were 3.9%, 3.2% and 9.8%, respectively. RGB histograms demonstrated significant differences in pixel value distribution between the four medical centers, as well as between the 3DHISTECH brand scanner when compared to the Philips and Hamamatsu brand scanners. Conclusions The results reported in this paper suggest that the algorithm-based decision support system has sufficient robustness to be applicable for clinical practice. In addition, the novel method used in its development – Hierarchial-Contexual Analysis (HCA) may be applicable to the development of algorithm-assisted tools in other diseases, for which available datasets are limited. Validation of any given algorithm-assisted support system should nonetheless include data from as many medical centers and scanner models as possible.https://doi.org/10.1186/s13000-024-01452-xHirschsprung's diseaseAlgorithmRobustnessMachine learningDigital pathology |
| spellingShingle | Ariel Greenberg Benzion Samueli Shai Farkash Yaniv Zohar Shahar Ish-Shalom Rami R. Hagege Dov Hershkovitz Algorithm-assisted diagnosis of Hirschsprung’s disease – evaluation of robustness and comparative image analysis on data from various labs and slide scanners Diagnostic Pathology Hirschsprung's disease Algorithm Robustness Machine learning Digital pathology |
| title | Algorithm-assisted diagnosis of Hirschsprung’s disease – evaluation of robustness and comparative image analysis on data from various labs and slide scanners |
| title_full | Algorithm-assisted diagnosis of Hirschsprung’s disease – evaluation of robustness and comparative image analysis on data from various labs and slide scanners |
| title_fullStr | Algorithm-assisted diagnosis of Hirschsprung’s disease – evaluation of robustness and comparative image analysis on data from various labs and slide scanners |
| title_full_unstemmed | Algorithm-assisted diagnosis of Hirschsprung’s disease – evaluation of robustness and comparative image analysis on data from various labs and slide scanners |
| title_short | Algorithm-assisted diagnosis of Hirschsprung’s disease – evaluation of robustness and comparative image analysis on data from various labs and slide scanners |
| title_sort | algorithm assisted diagnosis of hirschsprung s disease evaluation of robustness and comparative image analysis on data from various labs and slide scanners |
| topic | Hirschsprung's disease Algorithm Robustness Machine learning Digital pathology |
| url | https://doi.org/10.1186/s13000-024-01452-x |
| work_keys_str_mv | AT arielgreenberg algorithmassisteddiagnosisofhirschsprungsdiseaseevaluationofrobustnessandcomparativeimageanalysisondatafromvariouslabsandslidescanners AT benzionsamueli algorithmassisteddiagnosisofhirschsprungsdiseaseevaluationofrobustnessandcomparativeimageanalysisondatafromvariouslabsandslidescanners AT shaifarkash algorithmassisteddiagnosisofhirschsprungsdiseaseevaluationofrobustnessandcomparativeimageanalysisondatafromvariouslabsandslidescanners AT yanivzohar algorithmassisteddiagnosisofhirschsprungsdiseaseevaluationofrobustnessandcomparativeimageanalysisondatafromvariouslabsandslidescanners AT shaharishshalom algorithmassisteddiagnosisofhirschsprungsdiseaseevaluationofrobustnessandcomparativeimageanalysisondatafromvariouslabsandslidescanners AT ramirhagege algorithmassisteddiagnosisofhirschsprungsdiseaseevaluationofrobustnessandcomparativeimageanalysisondatafromvariouslabsandslidescanners AT dovhershkovitz algorithmassisteddiagnosisofhirschsprungsdiseaseevaluationofrobustnessandcomparativeimageanalysisondatafromvariouslabsandslidescanners |