Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning
This paper presents an automatic content-based image retrieval (CBIR) system for brain tumors on T1-weighted contrast-enhanced magnetic resonance images (CE-MRI). The key challenge in CBIR systems for MR images is the semantic gap between the low-level visual information captured by the MRI machine...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8611216/ |
| _version_ | 1818644542144905216 |
|---|---|
| author | Zar Nawab Khan Swati Qinghua Zhao Muhammad Kabir Farman Ali Zakir Ali Saeed Ahmed Jianfeng Lu |
| author_facet | Zar Nawab Khan Swati Qinghua Zhao Muhammad Kabir Farman Ali Zakir Ali Saeed Ahmed Jianfeng Lu |
| author_sort | Zar Nawab Khan Swati |
| collection | DOAJ |
| description | This paper presents an automatic content-based image retrieval (CBIR) system for brain tumors on T1-weighted contrast-enhanced magnetic resonance images (CE-MRI). The key challenge in CBIR systems for MR images is the semantic gap between the low-level visual information captured by the MRI machine and the high-level information perceived by the human evaluator. The traditional feature extraction methods focus only on low-level or high-level features and use some handcrafted features to reduce this gap. It is necessary to design a feature extraction framework to reduce this gap without using handcrafted features by encoding/combining low-level and high-level features. Deep learning is very powerful for feature representation that can depict low-level and high-level information completely and embed the phase of feature extraction in self-learning. Therefore, we propose a deep convolutional neural network VGG19-based novel feature extraction framework and apply closed-form metric learning to measure the similarity between the query image and database images. Furthermore, we adopt transfer learning and propose a block-wise fine-tuning strategy to enhance the retrieval performance. The extensive experiments are performed on a publicly available CE-MRI dataset that consists of three types of brain tumors (i.e., glioma, meningioma, and pituitary tumor) collected from 233 patients with a total of 3064 images across the axial, coronal, and sagittal views. Our method is more generic, as we do not use any handcrafted features; it requires minimal preprocessing, tested as robust on fivefold cross-validation, can achieve a fivefold mean average precision of 96.13%, and outperforms the state-of-the-art CBIR systems on the CE-MRI dataset. |
| first_indexed | 2024-12-17T00:16:30Z |
| format | Article |
| id | doaj.art-414c31c35d4f40c89482c378f1d6b494 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-17T00:16:30Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-414c31c35d4f40c89482c378f1d6b4942022-12-21T22:10:42ZengIEEEIEEE Access2169-35362019-01-017178091782210.1109/ACCESS.2019.28924558611216Content-Based Brain Tumor Retrieval for MR Images Using Transfer LearningZar Nawab Khan Swati0https://orcid.org/0000-0001-8474-9307Qinghua Zhao1Muhammad Kabir2https://orcid.org/0000-0002-2488-1653Farman Ali3Zakir Ali4Saeed Ahmed5Jianfeng Lu6School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaCollege of Information Engineering, Nanjing University of Finance and Economics, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaThis paper presents an automatic content-based image retrieval (CBIR) system for brain tumors on T1-weighted contrast-enhanced magnetic resonance images (CE-MRI). The key challenge in CBIR systems for MR images is the semantic gap between the low-level visual information captured by the MRI machine and the high-level information perceived by the human evaluator. The traditional feature extraction methods focus only on low-level or high-level features and use some handcrafted features to reduce this gap. It is necessary to design a feature extraction framework to reduce this gap without using handcrafted features by encoding/combining low-level and high-level features. Deep learning is very powerful for feature representation that can depict low-level and high-level information completely and embed the phase of feature extraction in self-learning. Therefore, we propose a deep convolutional neural network VGG19-based novel feature extraction framework and apply closed-form metric learning to measure the similarity between the query image and database images. Furthermore, we adopt transfer learning and propose a block-wise fine-tuning strategy to enhance the retrieval performance. The extensive experiments are performed on a publicly available CE-MRI dataset that consists of three types of brain tumors (i.e., glioma, meningioma, and pituitary tumor) collected from 233 patients with a total of 3064 images across the axial, coronal, and sagittal views. Our method is more generic, as we do not use any handcrafted features; it requires minimal preprocessing, tested as robust on fivefold cross-validation, can achieve a fivefold mean average precision of 96.13%, and outperforms the state-of-the-art CBIR systems on the CE-MRI dataset.https://ieeexplore.ieee.org/document/8611216/Brain tumor retrievalblock-wise fine-tuningclosed-form metric learningconvolutional neural networksfeature extractiontransfer learning |
| spellingShingle | Zar Nawab Khan Swati Qinghua Zhao Muhammad Kabir Farman Ali Zakir Ali Saeed Ahmed Jianfeng Lu Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning IEEE Access Brain tumor retrieval block-wise fine-tuning closed-form metric learning convolutional neural networks feature extraction transfer learning |
| title | Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning |
| title_full | Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning |
| title_fullStr | Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning |
| title_full_unstemmed | Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning |
| title_short | Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning |
| title_sort | content based brain tumor retrieval for mr images using transfer learning |
| topic | Brain tumor retrieval block-wise fine-tuning closed-form metric learning convolutional neural networks feature extraction transfer learning |
| url | https://ieeexplore.ieee.org/document/8611216/ |
| work_keys_str_mv | AT zarnawabkhanswati contentbasedbraintumorretrievalformrimagesusingtransferlearning AT qinghuazhao contentbasedbraintumorretrievalformrimagesusingtransferlearning AT muhammadkabir contentbasedbraintumorretrievalformrimagesusingtransferlearning AT farmanali contentbasedbraintumorretrievalformrimagesusingtransferlearning AT zakirali contentbasedbraintumorretrievalformrimagesusingtransferlearning AT saeedahmed contentbasedbraintumorretrievalformrimagesusingtransferlearning AT jianfenglu contentbasedbraintumorretrievalformrimagesusingtransferlearning |