Multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumors
Objective: We aim to establish a deep learning model called multimodal ultrasound fusion network (MUF-Net) based on gray-scale and contrast-enhanced ultrasound (CEUS) images for classifying benign and malignant solid renal tumors automatically and to compare the model’s performance with the assessme...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-09-01
|
| Series: | Frontiers in Molecular Biosciences |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fmolb.2022.982703/full |
| _version_ | 1811268771743006720 |
|---|---|
| author | Dongmei Zhu Dongmei Zhu Junyu Li Junyu Li Junyu Li Yan Li Ji Wu Lin Zhu Jian Li Zimo Wang Jinfeng Xu Fajin Dong Jun Cheng Jun Cheng Jun Cheng |
| author_facet | Dongmei Zhu Dongmei Zhu Junyu Li Junyu Li Junyu Li Yan Li Ji Wu Lin Zhu Jian Li Zimo Wang Jinfeng Xu Fajin Dong Jun Cheng Jun Cheng Jun Cheng |
| author_sort | Dongmei Zhu |
| collection | DOAJ |
| description | Objective: We aim to establish a deep learning model called multimodal ultrasound fusion network (MUF-Net) based on gray-scale and contrast-enhanced ultrasound (CEUS) images for classifying benign and malignant solid renal tumors automatically and to compare the model’s performance with the assessments by radiologists with different levels of experience.Methods: A retrospective study included the CEUS videos of 181 patients with solid renal tumors (81 benign and 100 malignant tumors) from June 2012 to June 2021. A total of 9794 B-mode and CEUS-mode images were cropped from the CEUS videos. The MUF-Net was proposed to combine gray-scale and CEUS images to differentiate benign and malignant solid renal tumors. In this network, two independent branches were designed to extract features from each of the two modalities, and the features were fused using adaptive weights. Finally, the network output a classification score based on the fused features. The model’s performance was evaluated using five-fold cross-validation and compared with the assessments of the two groups of radiologists with different levels of experience.Results: For the discrimination between benign and malignant solid renal tumors, the junior radiologist group, senior radiologist group, and MUF-Net achieved accuracy of 70.6%, 75.7%, and 80.0%, sensitivity of 89.3%, 95.9%, and 80.4%, specificity of 58.7%, 62.9%, and 79.1%, and area under the receiver operating characteristic curve of 0.740 (95% confidence internal (CI): 0.70–0.75), 0.794 (95% CI: 0.72–0.83), and 0.877 (95% CI: 0.83–0.93), respectively.Conclusion: The MUF-Net model can accurately classify benign and malignant solid renal tumors and achieve better performance than senior radiologists.Key points: The CEUS video data contain the entire tumor microcirculation perfusion characteristics. The proposed MUF-Net based on B-mode and CEUS-mode images can accurately distinguish between benign and malignant solid renal tumors with an area under the receiver operating characteristic curve of 0.877, which surpasses senior radiologists’ assessments by a large margin. |
| first_indexed | 2024-04-12T21:28:56Z |
| format | Article |
| id | doaj.art-40012b5f747141a3964a29553b9d3d2a |
| institution | Directory Open Access Journal |
| issn | 2296-889X |
| language | English |
| last_indexed | 2024-04-12T21:28:56Z |
| publishDate | 2022-09-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Molecular Biosciences |
| spelling | doaj.art-40012b5f747141a3964a29553b9d3d2a2022-12-22T03:16:05ZengFrontiers Media S.A.Frontiers in Molecular Biosciences2296-889X2022-09-01910.3389/fmolb.2022.982703982703Multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumorsDongmei Zhu0Dongmei Zhu1Junyu Li2Junyu Li3Junyu Li4Yan Li5Ji Wu6Lin Zhu7Jian Li8Zimo Wang9Jinfeng Xu10Fajin Dong11Jun Cheng12Jun Cheng13Jun Cheng14Department of Ultrasound, The Second Clinical Medical College, Jinan University, Shenzhen, ChinaDepartment of Ultrasound, The Affiliated Nanchong Central Hospital of North Sichuan Medical College, Nanchong, ChinaNational-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, ChinaMedical Ultrasound Image Computing (MUSIC) Laboratory, Shenzhen University, Shenzhen, ChinaMarshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, ChinaDepartment of Ultrasound, The Affiliated Nanchong Central Hospital of North Sichuan Medical College, Nanchong, ChinaDepartment of Urology Surgery, The Affiliated Nanchong Central Hospital of North Sichuan Medical College, Nanchong, ChinaDepartment of Ultrasound, The Affiliated Nanchong Central Hospital of North Sichuan Medical College, Nanchong, ChinaDepartment of Ultrasound, The Second Clinical Medical College, Jinan University, Shenzhen, ChinaDepartment of Ultrasound, The Second Clinical Medical College, Jinan University, Shenzhen, ChinaDepartment of Ultrasound, The Second Clinical Medical College, Jinan University, Shenzhen, ChinaDepartment of Ultrasound, The Second Clinical Medical College, Jinan University, Shenzhen, ChinaNational-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, ChinaMedical Ultrasound Image Computing (MUSIC) Laboratory, Shenzhen University, Shenzhen, ChinaMarshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, ChinaObjective: We aim to establish a deep learning model called multimodal ultrasound fusion network (MUF-Net) based on gray-scale and contrast-enhanced ultrasound (CEUS) images for classifying benign and malignant solid renal tumors automatically and to compare the model’s performance with the assessments by radiologists with different levels of experience.Methods: A retrospective study included the CEUS videos of 181 patients with solid renal tumors (81 benign and 100 malignant tumors) from June 2012 to June 2021. A total of 9794 B-mode and CEUS-mode images were cropped from the CEUS videos. The MUF-Net was proposed to combine gray-scale and CEUS images to differentiate benign and malignant solid renal tumors. In this network, two independent branches were designed to extract features from each of the two modalities, and the features were fused using adaptive weights. Finally, the network output a classification score based on the fused features. The model’s performance was evaluated using five-fold cross-validation and compared with the assessments of the two groups of radiologists with different levels of experience.Results: For the discrimination between benign and malignant solid renal tumors, the junior radiologist group, senior radiologist group, and MUF-Net achieved accuracy of 70.6%, 75.7%, and 80.0%, sensitivity of 89.3%, 95.9%, and 80.4%, specificity of 58.7%, 62.9%, and 79.1%, and area under the receiver operating characteristic curve of 0.740 (95% confidence internal (CI): 0.70–0.75), 0.794 (95% CI: 0.72–0.83), and 0.877 (95% CI: 0.83–0.93), respectively.Conclusion: The MUF-Net model can accurately classify benign and malignant solid renal tumors and achieve better performance than senior radiologists.Key points: The CEUS video data contain the entire tumor microcirculation perfusion characteristics. The proposed MUF-Net based on B-mode and CEUS-mode images can accurately distinguish between benign and malignant solid renal tumors with an area under the receiver operating characteristic curve of 0.877, which surpasses senior radiologists’ assessments by a large margin.https://www.frontiersin.org/articles/10.3389/fmolb.2022.982703/fullrenal tumorartificial intelligenceclassificationdeep learningcontrast-enhanced ultrasound |
| spellingShingle | Dongmei Zhu Dongmei Zhu Junyu Li Junyu Li Junyu Li Yan Li Ji Wu Lin Zhu Jian Li Zimo Wang Jinfeng Xu Fajin Dong Jun Cheng Jun Cheng Jun Cheng Multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumors Frontiers in Molecular Biosciences renal tumor artificial intelligence classification deep learning contrast-enhanced ultrasound |
| title | Multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumors |
| title_full | Multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumors |
| title_fullStr | Multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumors |
| title_full_unstemmed | Multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumors |
| title_short | Multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumors |
| title_sort | multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumors |
| topic | renal tumor artificial intelligence classification deep learning contrast-enhanced ultrasound |
| url | https://www.frontiersin.org/articles/10.3389/fmolb.2022.982703/full |
| work_keys_str_mv | AT dongmeizhu multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT dongmeizhu multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT junyuli multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT junyuli multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT junyuli multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT yanli multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT jiwu multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT linzhu multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT jianli multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT zimowang multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT jinfengxu multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT fajindong multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT juncheng multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT juncheng multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors AT juncheng multimodalultrasoundfusionnetworkfordifferentiatingbetweenbenignandmalignantsolidrenaltumors |