A generalized deep learning framework for whole-slide image segmentation and analysis
Abstract Histopathology tissue analysis is considered the gold standard in cancer diagnosis and prognosis. Whole-slide imaging (WSI), i.e., the scanning and digitization of entire histology slides, are now being adopted across the world in pathology labs. Trained histopathologists can provide an acc...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2021-06-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-021-90444-8 |
| _version_ | 1818683289147277312 |
|---|---|
| author | Mahendra Khened Avinash Kori Haran Rajkumar Ganapathy Krishnamurthi Balaji Srinivasan |
| author_facet | Mahendra Khened Avinash Kori Haran Rajkumar Ganapathy Krishnamurthi Balaji Srinivasan |
| author_sort | Mahendra Khened |
| collection | DOAJ |
| description | Abstract Histopathology tissue analysis is considered the gold standard in cancer diagnosis and prognosis. Whole-slide imaging (WSI), i.e., the scanning and digitization of entire histology slides, are now being adopted across the world in pathology labs. Trained histopathologists can provide an accurate diagnosis of biopsy specimens based on WSI data. Given the dimensionality of WSIs and the increase in the number of potential cancer cases, analyzing these images is a time-consuming process. Automated segmentation of tumorous tissue helps in elevating the precision, speed, and reproducibility of research. In the recent past, deep learning-based techniques have provided state-of-the-art results in a wide variety of image analysis tasks, including the analysis of digitized slides. However, deep learning-based solutions pose many technical challenges, including the large size of WSI data, heterogeneity in images, and complexity of features. In this study, we propose a generalized deep learning-based framework for histopathology tissue analysis to address these challenges. Our framework is, in essence, a sequence of individual techniques in the preprocessing-training-inference pipeline which, in conjunction, improve the efficiency and the generalizability of the analysis. The combination of techniques we have introduced includes an ensemble segmentation model, division of the WSI into smaller overlapping patches while addressing class imbalances, efficient techniques for inference, and an efficient, patch-based uncertainty estimation framework. Our ensemble consists of DenseNet-121, Inception-ResNet-V2, and DeeplabV3Plus, where all the networks were trained end to end for every task. We demonstrate the efficacy and improved generalizability of our framework by evaluating it on a variety of histopathology tasks including breast cancer metastases (CAMELYON), colon cancer (DigestPath), and liver cancer (PAIP). Our proposed framework has state-of-the-art performance across all these tasks and is ranked within the top 5 currently for the challenges based on these datasets. The entire framework along with the trained models and the related documentation are made freely available at GitHub and PyPi. Our framework is expected to aid histopathologists in accurate and efficient initial diagnosis. Moreover, the estimated uncertainty maps will help clinicians to make informed decisions and further treatment planning or analysis. |
| first_indexed | 2024-12-17T10:32:22Z |
| format | Article |
| id | doaj.art-41e5df922c4a416987a654bf7f568fc6 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-12-17T10:32:22Z |
| publishDate | 2021-06-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-41e5df922c4a416987a654bf7f568fc62022-12-21T21:52:29ZengNature PortfolioScientific Reports2045-23222021-06-0111111410.1038/s41598-021-90444-8A generalized deep learning framework for whole-slide image segmentation and analysisMahendra Khened0Avinash Kori1Haran Rajkumar2Ganapathy Krishnamurthi3Balaji Srinivasan4Department of Engineering Design, Indian Institute of Technology MadrasDepartment of Engineering Design, Indian Institute of Technology MadrasDepartment of Engineering Design, Indian Institute of Technology MadrasDepartment of Engineering Design, Indian Institute of Technology MadrasDepartment of Mechanical Engineering, Indian Institute of Technology MadrasAbstract Histopathology tissue analysis is considered the gold standard in cancer diagnosis and prognosis. Whole-slide imaging (WSI), i.e., the scanning and digitization of entire histology slides, are now being adopted across the world in pathology labs. Trained histopathologists can provide an accurate diagnosis of biopsy specimens based on WSI data. Given the dimensionality of WSIs and the increase in the number of potential cancer cases, analyzing these images is a time-consuming process. Automated segmentation of tumorous tissue helps in elevating the precision, speed, and reproducibility of research. In the recent past, deep learning-based techniques have provided state-of-the-art results in a wide variety of image analysis tasks, including the analysis of digitized slides. However, deep learning-based solutions pose many technical challenges, including the large size of WSI data, heterogeneity in images, and complexity of features. In this study, we propose a generalized deep learning-based framework for histopathology tissue analysis to address these challenges. Our framework is, in essence, a sequence of individual techniques in the preprocessing-training-inference pipeline which, in conjunction, improve the efficiency and the generalizability of the analysis. The combination of techniques we have introduced includes an ensemble segmentation model, division of the WSI into smaller overlapping patches while addressing class imbalances, efficient techniques for inference, and an efficient, patch-based uncertainty estimation framework. Our ensemble consists of DenseNet-121, Inception-ResNet-V2, and DeeplabV3Plus, where all the networks were trained end to end for every task. We demonstrate the efficacy and improved generalizability of our framework by evaluating it on a variety of histopathology tasks including breast cancer metastases (CAMELYON), colon cancer (DigestPath), and liver cancer (PAIP). Our proposed framework has state-of-the-art performance across all these tasks and is ranked within the top 5 currently for the challenges based on these datasets. The entire framework along with the trained models and the related documentation are made freely available at GitHub and PyPi. Our framework is expected to aid histopathologists in accurate and efficient initial diagnosis. Moreover, the estimated uncertainty maps will help clinicians to make informed decisions and further treatment planning or analysis.https://doi.org/10.1038/s41598-021-90444-8 |
| spellingShingle | Mahendra Khened Avinash Kori Haran Rajkumar Ganapathy Krishnamurthi Balaji Srinivasan A generalized deep learning framework for whole-slide image segmentation and analysis Scientific Reports |
| title | A generalized deep learning framework for whole-slide image segmentation and analysis |
| title_full | A generalized deep learning framework for whole-slide image segmentation and analysis |
| title_fullStr | A generalized deep learning framework for whole-slide image segmentation and analysis |
| title_full_unstemmed | A generalized deep learning framework for whole-slide image segmentation and analysis |
| title_short | A generalized deep learning framework for whole-slide image segmentation and analysis |
| title_sort | generalized deep learning framework for whole slide image segmentation and analysis |
| url | https://doi.org/10.1038/s41598-021-90444-8 |
| work_keys_str_mv | AT mahendrakhened ageneralizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis AT avinashkori ageneralizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis AT haranrajkumar ageneralizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis AT ganapathykrishnamurthi ageneralizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis AT balajisrinivasan ageneralizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis AT mahendrakhened generalizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis AT avinashkori generalizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis AT haranrajkumar generalizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis AT ganapathykrishnamurthi generalizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis AT balajisrinivasan generalizeddeeplearningframeworkforwholeslideimagesegmentationandanalysis |