Weakly Supervised Tumor Detection in PET Using Class Response for Treatment Outcome Prediction
It is proven that radiomic characteristics extracted from the tumor region are predictive. The first step in radiomic analysis is the segmentation of the lesion. However, this task is time consuming and requires a highly trained physician. This process could be automated using computer-aided detecti...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-05-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/8/5/130 |
| _version_ | 1797498818499969024 |
|---|---|
| author | Amine Amyar Romain Modzelewski Pierre Vera Vincent Morard Su Ruan |
| author_facet | Amine Amyar Romain Modzelewski Pierre Vera Vincent Morard Su Ruan |
| author_sort | Amine Amyar |
| collection | DOAJ |
| description | It is proven that radiomic characteristics extracted from the tumor region are predictive. The first step in radiomic analysis is the segmentation of the lesion. However, this task is time consuming and requires a highly trained physician. This process could be automated using computer-aided detection (CAD) tools. Current state-of-the-art methods are trained in a supervised learning setting, which requires a lot of data that are usually not available in the medical imaging field. The challenge is to train one model to segment different types of tumors with only a weak segmentation ground truth. In this work, we propose a prediction framework including a 3D tumor segmentation in positron emission tomography (PET) images, based on a weakly supervised deep learning method, and an outcome prediction based on a 3D-CNN classifier applied to the segmented tumor regions. The key step is to locate the tumor in 3D. We propose to (1) calculate two maximum intensity projection (MIP) images from 3D PET images in two directions, (2) classify the MIP images into different types of cancers, (3) generate the class activation maps through a multitask learning approach with a weak prior knowledge, and (4) segment the 3D tumor region from the two 2D activation maps with a proposed new loss function for the multitask. The proposed approach achieves state-of-the-art prediction results with a small data set and with a weak segmentation ground truth. Our model was tested and validated for treatment response and survival in lung and esophageal cancers on 195 patients, with an area under the receiver operating characteristic curve (AUC) of 67% and 59%, respectively, and a dice coefficient of 73% and 0.77% for tumor segmentation. |
| first_indexed | 2024-03-10T03:38:48Z |
| format | Article |
| id | doaj.art-8e71e8fead594985a33052c9cc101ad6 |
| institution | Directory Open Access Journal |
| issn | 2313-433X |
| language | English |
| last_indexed | 2024-03-10T03:38:48Z |
| publishDate | 2022-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj.art-8e71e8fead594985a33052c9cc101ad62023-11-23T11:38:04ZengMDPI AGJournal of Imaging2313-433X2022-05-018513010.3390/jimaging8050130Weakly Supervised Tumor Detection in PET Using Class Response for Treatment Outcome PredictionAmine Amyar0Romain Modzelewski1Pierre Vera2Vincent Morard3Su Ruan4General Electric Healthcare, 78530 Buc, FranceLITIS-EA4108-Quantif, University of Rouen, 76800 Rouen, FranceLITIS-EA4108-Quantif, University of Rouen, 76800 Rouen, FranceGeneral Electric Healthcare, 78530 Buc, FranceLITIS-EA4108-Quantif, University of Rouen, 76800 Rouen, FranceIt is proven that radiomic characteristics extracted from the tumor region are predictive. The first step in radiomic analysis is the segmentation of the lesion. However, this task is time consuming and requires a highly trained physician. This process could be automated using computer-aided detection (CAD) tools. Current state-of-the-art methods are trained in a supervised learning setting, which requires a lot of data that are usually not available in the medical imaging field. The challenge is to train one model to segment different types of tumors with only a weak segmentation ground truth. In this work, we propose a prediction framework including a 3D tumor segmentation in positron emission tomography (PET) images, based on a weakly supervised deep learning method, and an outcome prediction based on a 3D-CNN classifier applied to the segmented tumor regions. The key step is to locate the tumor in 3D. We propose to (1) calculate two maximum intensity projection (MIP) images from 3D PET images in two directions, (2) classify the MIP images into different types of cancers, (3) generate the class activation maps through a multitask learning approach with a weak prior knowledge, and (4) segment the 3D tumor region from the two 2D activation maps with a proposed new loss function for the multitask. The proposed approach achieves state-of-the-art prediction results with a small data set and with a weak segmentation ground truth. Our model was tested and validated for treatment response and survival in lung and esophageal cancers on 195 patients, with an area under the receiver operating characteristic curve (AUC) of 67% and 59%, respectively, and a dice coefficient of 73% and 0.77% for tumor segmentation.https://www.mdpi.com/2313-433X/8/5/130weakly supervised learningclass activation mapstumor detectionradiomicsimage classificationimage segmentation |
| spellingShingle | Amine Amyar Romain Modzelewski Pierre Vera Vincent Morard Su Ruan Weakly Supervised Tumor Detection in PET Using Class Response for Treatment Outcome Prediction Journal of Imaging weakly supervised learning class activation maps tumor detection radiomics image classification image segmentation |
| title | Weakly Supervised Tumor Detection in PET Using Class Response for Treatment Outcome Prediction |
| title_full | Weakly Supervised Tumor Detection in PET Using Class Response for Treatment Outcome Prediction |
| title_fullStr | Weakly Supervised Tumor Detection in PET Using Class Response for Treatment Outcome Prediction |
| title_full_unstemmed | Weakly Supervised Tumor Detection in PET Using Class Response for Treatment Outcome Prediction |
| title_short | Weakly Supervised Tumor Detection in PET Using Class Response for Treatment Outcome Prediction |
| title_sort | weakly supervised tumor detection in pet using class response for treatment outcome prediction |
| topic | weakly supervised learning class activation maps tumor detection radiomics image classification image segmentation |
| url | https://www.mdpi.com/2313-433X/8/5/130 |
| work_keys_str_mv | AT amineamyar weaklysupervisedtumordetectioninpetusingclassresponsefortreatmentoutcomeprediction AT romainmodzelewski weaklysupervisedtumordetectioninpetusingclassresponsefortreatmentoutcomeprediction AT pierrevera weaklysupervisedtumordetectioninpetusingclassresponsefortreatmentoutcomeprediction AT vincentmorard weaklysupervisedtumordetectioninpetusingclassresponsefortreatmentoutcomeprediction AT suruan weaklysupervisedtumordetectioninpetusingclassresponsefortreatmentoutcomeprediction |