A new generative adversarial network for medical images super resolution
Abstract For medical image analysis, there is always an immense need for rich details in an image. Typically, the diagnosis will be served best if the fine details in the image are retained and the image is available in high resolution. In medical imaging, acquiring high-resolution images is challen...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-06-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-13658-4 |
| _version_ | 1818469287076036608 |
|---|---|
| author | Waqar Ahmad Hazrat Ali Zubair Shah Shoaib Azmat |
| author_facet | Waqar Ahmad Hazrat Ali Zubair Shah Shoaib Azmat |
| author_sort | Waqar Ahmad |
| collection | DOAJ |
| description | Abstract For medical image analysis, there is always an immense need for rich details in an image. Typically, the diagnosis will be served best if the fine details in the image are retained and the image is available in high resolution. In medical imaging, acquiring high-resolution images is challenging and costly as it requires sophisticated and expensive instruments, trained human resources, and often causes operation delays. Deep learning based super resolution techniques can help us to extract rich details from a low-resolution image acquired using the existing devices. In this paper, we propose a new Generative Adversarial Network (GAN) based architecture for medical images, which maps low-resolution medical images to high-resolution images. The proposed architecture is divided into three steps. In the first step, we use a multi-path architecture to extract shallow features on multiple scales instead of single scale. In the second step, we use a ResNet34 architecture to extract deep features and upscale the features map by a factor of two. In the third step, we extract features of the upscaled version of the image using a residual connection-based mini-CNN and again upscale the feature map by a factor of two. The progressive upscaling overcomes the limitation for previous methods in generating true colors. Finally, we use a reconstruction convolutional layer to map back the upscaled features to a high-resolution image. Our addition of an extra loss term helps in overcoming large errors, thus, generating more realistic and smooth images. We evaluate the proposed architecture on four different medical image modalities: (1) the DRIVE and STARE datasets of retinal fundoscopy images, (2) the BraTS dataset of brain MRI, (3) the ISIC skin cancer dataset of dermoscopy images, and (4) the CAMUS dataset of cardiac ultrasound images. The proposed architecture achieves superior accuracy compared to other state-of-the-art super-resolution architectures. |
| first_indexed | 2024-04-13T21:22:19Z |
| format | Article |
| id | doaj.art-923aa14361cd4b5b881a4a8f31e4e71a |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-13T21:22:19Z |
| publishDate | 2022-06-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-923aa14361cd4b5b881a4a8f31e4e71a2022-12-22T02:29:27ZengNature PortfolioScientific Reports2045-23222022-06-0112112010.1038/s41598-022-13658-4A new generative adversarial network for medical images super resolutionWaqar Ahmad0Hazrat Ali1Zubair Shah2Shoaib Azmat3Department of Electrical and Computer Engineering, COMSATS University IslamabadCollege of Science and Engineering, Hamad Bin Khalifa UniversityCollege of Science and Engineering, Hamad Bin Khalifa UniversityDepartment of Electrical and Computer Engineering, COMSATS University IslamabadAbstract For medical image analysis, there is always an immense need for rich details in an image. Typically, the diagnosis will be served best if the fine details in the image are retained and the image is available in high resolution. In medical imaging, acquiring high-resolution images is challenging and costly as it requires sophisticated and expensive instruments, trained human resources, and often causes operation delays. Deep learning based super resolution techniques can help us to extract rich details from a low-resolution image acquired using the existing devices. In this paper, we propose a new Generative Adversarial Network (GAN) based architecture for medical images, which maps low-resolution medical images to high-resolution images. The proposed architecture is divided into three steps. In the first step, we use a multi-path architecture to extract shallow features on multiple scales instead of single scale. In the second step, we use a ResNet34 architecture to extract deep features and upscale the features map by a factor of two. In the third step, we extract features of the upscaled version of the image using a residual connection-based mini-CNN and again upscale the feature map by a factor of two. The progressive upscaling overcomes the limitation for previous methods in generating true colors. Finally, we use a reconstruction convolutional layer to map back the upscaled features to a high-resolution image. Our addition of an extra loss term helps in overcoming large errors, thus, generating more realistic and smooth images. We evaluate the proposed architecture on four different medical image modalities: (1) the DRIVE and STARE datasets of retinal fundoscopy images, (2) the BraTS dataset of brain MRI, (3) the ISIC skin cancer dataset of dermoscopy images, and (4) the CAMUS dataset of cardiac ultrasound images. The proposed architecture achieves superior accuracy compared to other state-of-the-art super-resolution architectures.https://doi.org/10.1038/s41598-022-13658-4 |
| spellingShingle | Waqar Ahmad Hazrat Ali Zubair Shah Shoaib Azmat A new generative adversarial network for medical images super resolution Scientific Reports |
| title | A new generative adversarial network for medical images super resolution |
| title_full | A new generative adversarial network for medical images super resolution |
| title_fullStr | A new generative adversarial network for medical images super resolution |
| title_full_unstemmed | A new generative adversarial network for medical images super resolution |
| title_short | A new generative adversarial network for medical images super resolution |
| title_sort | new generative adversarial network for medical images super resolution |
| url | https://doi.org/10.1038/s41598-022-13658-4 |
| work_keys_str_mv | AT waqarahmad anewgenerativeadversarialnetworkformedicalimagessuperresolution AT hazratali anewgenerativeadversarialnetworkformedicalimagessuperresolution AT zubairshah anewgenerativeadversarialnetworkformedicalimagessuperresolution AT shoaibazmat anewgenerativeadversarialnetworkformedicalimagessuperresolution AT waqarahmad newgenerativeadversarialnetworkformedicalimagessuperresolution AT hazratali newgenerativeadversarialnetworkformedicalimagessuperresolution AT zubairshah newgenerativeadversarialnetworkformedicalimagessuperresolution AT shoaibazmat newgenerativeadversarialnetworkformedicalimagessuperresolution |