Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays
Deep learning (DL) methods have demonstrated superior performance in medical image segmentation tasks. However, selecting a loss function that conforms to the data characteristics is critical for optimal performance. Further, the direct use of traditional DL models does not provide a measure of unce...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Biomedicines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-9059/10/6/1323 |
| _version_ | 1827662167010181120 |
|---|---|
| author | Sivaramakrishnan Rajaraman Ghada Zamzmi Feng Yang Zhiyun Xue Stefan Jaeger Sameer K. Antani |
| author_facet | Sivaramakrishnan Rajaraman Ghada Zamzmi Feng Yang Zhiyun Xue Stefan Jaeger Sameer K. Antani |
| author_sort | Sivaramakrishnan Rajaraman |
| collection | DOAJ |
| description | Deep learning (DL) methods have demonstrated superior performance in medical image segmentation tasks. However, selecting a loss function that conforms to the data characteristics is critical for optimal performance. Further, the direct use of traditional DL models does not provide a measure of uncertainty in predictions. Even high-quality automated predictions for medical diagnostic applications demand uncertainty quantification to gain user trust. In this study, we aim to investigate the benefits of (i) selecting an appropriate loss function and (ii) quantifying uncertainty in predictions using a VGG16-based-U-Net model with the Monto–Carlo (MCD) Dropout method for segmenting Tuberculosis (TB)-consistent findings in frontal chest X-rays (CXRs). We determine an optimal uncertainty threshold based on several uncertainty-related metrics. This threshold is used to select and refer highly uncertain cases to an expert. Experimental results demonstrate that (i) the model trained with a modified Focal Tversky loss function delivered superior segmentation performance (mean average precision (mAP): 0.5710, 95% confidence interval (CI): (0.4021,0.7399)), (ii) the model with 30 MC forward passes during inference further improved and stabilized performance (mAP: 0.5721, 95% CI: (0.4032,0.7410), and (iii) an uncertainty threshold of 0.7 is observed to be optimal to refer highly uncertain cases. |
| first_indexed | 2024-03-10T00:21:19Z |
| format | Article |
| id | doaj.art-eb7c81bdd2234233a48a8578aca83dc6 |
| institution | Directory Open Access Journal |
| issn | 2227-9059 |
| language | English |
| last_indexed | 2024-03-10T00:21:19Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biomedicines |
| spelling | doaj.art-eb7c81bdd2234233a48a8578aca83dc62023-11-23T15:42:48ZengMDPI AGBiomedicines2227-90592022-06-01106132310.3390/biomedicines10061323Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-raysSivaramakrishnan Rajaraman0Ghada Zamzmi1Feng Yang2Zhiyun Xue3Stefan Jaeger4Sameer K. Antani5National Library of Medicine, National Institutes of Health, Bethesda, MD 20892, USANational Library of Medicine, National Institutes of Health, Bethesda, MD 20892, USANational Library of Medicine, National Institutes of Health, Bethesda, MD 20892, USANational Library of Medicine, National Institutes of Health, Bethesda, MD 20892, USANational Library of Medicine, National Institutes of Health, Bethesda, MD 20892, USANational Library of Medicine, National Institutes of Health, Bethesda, MD 20892, USADeep learning (DL) methods have demonstrated superior performance in medical image segmentation tasks. However, selecting a loss function that conforms to the data characteristics is critical for optimal performance. Further, the direct use of traditional DL models does not provide a measure of uncertainty in predictions. Even high-quality automated predictions for medical diagnostic applications demand uncertainty quantification to gain user trust. In this study, we aim to investigate the benefits of (i) selecting an appropriate loss function and (ii) quantifying uncertainty in predictions using a VGG16-based-U-Net model with the Monto–Carlo (MCD) Dropout method for segmenting Tuberculosis (TB)-consistent findings in frontal chest X-rays (CXRs). We determine an optimal uncertainty threshold based on several uncertainty-related metrics. This threshold is used to select and refer highly uncertain cases to an expert. Experimental results demonstrate that (i) the model trained with a modified Focal Tversky loss function delivered superior segmentation performance (mean average precision (mAP): 0.5710, 95% confidence interval (CI): (0.4021,0.7399)), (ii) the model with 30 MC forward passes during inference further improved and stabilized performance (mAP: 0.5721, 95% CI: (0.4032,0.7410), and (iii) an uncertainty threshold of 0.7 is observed to be optimal to refer highly uncertain cases.https://www.mdpi.com/2227-9059/10/6/1323chest X-rayuncertaintyuncertainty quantificationdeep learningmedical image segmentationtuberculosis |
| spellingShingle | Sivaramakrishnan Rajaraman Ghada Zamzmi Feng Yang Zhiyun Xue Stefan Jaeger Sameer K. Antani Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays Biomedicines chest X-ray uncertainty uncertainty quantification deep learning medical image segmentation tuberculosis |
| title | Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays |
| title_full | Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays |
| title_fullStr | Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays |
| title_full_unstemmed | Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays |
| title_short | Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays |
| title_sort | uncertainty quantification in segmenting tuberculosis consistent findings in frontal chest x rays |
| topic | chest X-ray uncertainty uncertainty quantification deep learning medical image segmentation tuberculosis |
| url | https://www.mdpi.com/2227-9059/10/6/1323 |
| work_keys_str_mv | AT sivaramakrishnanrajaraman uncertaintyquantificationinsegmentingtuberculosisconsistentfindingsinfrontalchestxrays AT ghadazamzmi uncertaintyquantificationinsegmentingtuberculosisconsistentfindingsinfrontalchestxrays AT fengyang uncertaintyquantificationinsegmentingtuberculosisconsistentfindingsinfrontalchestxrays AT zhiyunxue uncertaintyquantificationinsegmentingtuberculosisconsistentfindingsinfrontalchestxrays AT stefanjaeger uncertaintyquantificationinsegmentingtuberculosisconsistentfindingsinfrontalchestxrays AT sameerkantani uncertaintyquantificationinsegmentingtuberculosisconsistentfindingsinfrontalchestxrays |