Fusing hand-crafted and deep-learning features in a convolutional neural network model to identify prostate cancer in pathology images
Prostate cancer can be diagnosed by prostate biopsy using transectal ultrasound guidance. The high number of pathology images from biopsy tissues is a burden on pathologists, and analysis is subjective and susceptible to inter-rater variability. The use of machine learning techniques could make pros...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-09-01
|
| Series: | Frontiers in Oncology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fonc.2022.994950/full |
| _version_ | 1811203791487238144 |
|---|---|
| author | Xinrui Huang Zhaotong Li Minghui Zhang Song Gao |
| author_facet | Xinrui Huang Zhaotong Li Minghui Zhang Song Gao |
| author_sort | Xinrui Huang |
| collection | DOAJ |
| description | Prostate cancer can be diagnosed by prostate biopsy using transectal ultrasound guidance. The high number of pathology images from biopsy tissues is a burden on pathologists, and analysis is subjective and susceptible to inter-rater variability. The use of machine learning techniques could make prostate histopathology diagnostics more precise, consistent, and efficient overall. This paper presents a new classification fusion network model that was created by fusing eight advanced image features: seven hand-crafted features and one deep-learning feature. These features are the scale-invariant feature transform (SIFT), speeded up robust feature (SURF), oriented features from accelerated segment test (FAST) and rotated binary robust independent elementary features (BRIEF) (ORB) of local features, shape and texture features of the cell nuclei, the histogram of oriented gradients (HOG) feature of the cavities, a color feature, and a convolution deep-learning feature. Matching, integrated, and fusion networks are the three essential components of the proposed deep-learning network. The integrated network consists of both a backbone and an additional network. When classifying 1100 prostate pathology images using this fusion network with different backbones (ResNet-18/50, VGG-11/16, and DenseNet-121/201), we discovered that the proposed model with the ResNet-18 backbone achieved the best performance in terms of the accuracy (95.54%), specificity (93.64%), and sensitivity (97.27%) as well as the area under the receiver operating characteristic curve (98.34%). However, each of the assessment criteria for these separate features had a value lower than 90%, which demonstrates that the suggested model combines differently derived characteristics in an effective manner. Moreover, a Grad-CAM++ heatmap was used to observe the differences between the proposed model and ResNet-18 in terms of the regions of interest. This map showed that the proposed model was better at focusing on cancerous cells than ResNet-18. Hence, the proposed classification fusion network, which combines hand-crafted features and a deep-learning feature, is useful for computer-aided diagnoses based on pathology images of prostate cancer. Because of the similarities in the feature engineering and deep learning for different types of pathology images, the proposed method could be used for other pathology images, such as those of breast, thyroid cancer. |
| first_indexed | 2024-04-12T03:02:14Z |
| format | Article |
| id | doaj.art-d3523b0ed5d045e5b91bb8a965d0100b |
| institution | Directory Open Access Journal |
| issn | 2234-943X |
| language | English |
| last_indexed | 2024-04-12T03:02:14Z |
| publishDate | 2022-09-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Oncology |
| spelling | doaj.art-d3523b0ed5d045e5b91bb8a965d0100b2022-12-22T03:50:38ZengFrontiers Media S.A.Frontiers in Oncology2234-943X2022-09-011210.3389/fonc.2022.994950994950Fusing hand-crafted and deep-learning features in a convolutional neural network model to identify prostate cancer in pathology imagesXinrui Huang0Zhaotong Li1Minghui Zhang2Song Gao3Department of Biochemistry and Biophysics, School of Basic Medical Sciences, Peking University, Beijing, ChinaInstitute of Medical Technology, Health Science Center, Peking University, Beijing, ChinaDepartment of Pathology, Guangdong Provincial People’s Hospital, Guangzhou, ChinaInstitute of Medical Technology, Health Science Center, Peking University, Beijing, ChinaProstate cancer can be diagnosed by prostate biopsy using transectal ultrasound guidance. The high number of pathology images from biopsy tissues is a burden on pathologists, and analysis is subjective and susceptible to inter-rater variability. The use of machine learning techniques could make prostate histopathology diagnostics more precise, consistent, and efficient overall. This paper presents a new classification fusion network model that was created by fusing eight advanced image features: seven hand-crafted features and one deep-learning feature. These features are the scale-invariant feature transform (SIFT), speeded up robust feature (SURF), oriented features from accelerated segment test (FAST) and rotated binary robust independent elementary features (BRIEF) (ORB) of local features, shape and texture features of the cell nuclei, the histogram of oriented gradients (HOG) feature of the cavities, a color feature, and a convolution deep-learning feature. Matching, integrated, and fusion networks are the three essential components of the proposed deep-learning network. The integrated network consists of both a backbone and an additional network. When classifying 1100 prostate pathology images using this fusion network with different backbones (ResNet-18/50, VGG-11/16, and DenseNet-121/201), we discovered that the proposed model with the ResNet-18 backbone achieved the best performance in terms of the accuracy (95.54%), specificity (93.64%), and sensitivity (97.27%) as well as the area under the receiver operating characteristic curve (98.34%). However, each of the assessment criteria for these separate features had a value lower than 90%, which demonstrates that the suggested model combines differently derived characteristics in an effective manner. Moreover, a Grad-CAM++ heatmap was used to observe the differences between the proposed model and ResNet-18 in terms of the regions of interest. This map showed that the proposed model was better at focusing on cancerous cells than ResNet-18. Hence, the proposed classification fusion network, which combines hand-crafted features and a deep-learning feature, is useful for computer-aided diagnoses based on pathology images of prostate cancer. Because of the similarities in the feature engineering and deep learning for different types of pathology images, the proposed method could be used for other pathology images, such as those of breast, thyroid cancer.https://www.frontiersin.org/articles/10.3389/fonc.2022.994950/fullprostate cancerpathology imageconvolutional neural network modelfeature fusionclassification |
| spellingShingle | Xinrui Huang Zhaotong Li Minghui Zhang Song Gao Fusing hand-crafted and deep-learning features in a convolutional neural network model to identify prostate cancer in pathology images Frontiers in Oncology prostate cancer pathology image convolutional neural network model feature fusion classification |
| title | Fusing hand-crafted and deep-learning features in a convolutional neural network model to identify prostate cancer in pathology images |
| title_full | Fusing hand-crafted and deep-learning features in a convolutional neural network model to identify prostate cancer in pathology images |
| title_fullStr | Fusing hand-crafted and deep-learning features in a convolutional neural network model to identify prostate cancer in pathology images |
| title_full_unstemmed | Fusing hand-crafted and deep-learning features in a convolutional neural network model to identify prostate cancer in pathology images |
| title_short | Fusing hand-crafted and deep-learning features in a convolutional neural network model to identify prostate cancer in pathology images |
| title_sort | fusing hand crafted and deep learning features in a convolutional neural network model to identify prostate cancer in pathology images |
| topic | prostate cancer pathology image convolutional neural network model feature fusion classification |
| url | https://www.frontiersin.org/articles/10.3389/fonc.2022.994950/full |
| work_keys_str_mv | AT xinruihuang fusinghandcraftedanddeeplearningfeaturesinaconvolutionalneuralnetworkmodeltoidentifyprostatecancerinpathologyimages AT zhaotongli fusinghandcraftedanddeeplearningfeaturesinaconvolutionalneuralnetworkmodeltoidentifyprostatecancerinpathologyimages AT minghuizhang fusinghandcraftedanddeeplearningfeaturesinaconvolutionalneuralnetworkmodeltoidentifyprostatecancerinpathologyimages AT songgao fusinghandcraftedanddeeplearningfeaturesinaconvolutionalneuralnetworkmodeltoidentifyprostatecancerinpathologyimages |