MSFSA-GAN: Multi-Scale Fusion Self Attention Generative Adversarial Network for Single Image Deraining
Bad weather such as rainy days will seriously affect the image quality and the accuracy of visual processing algorithm. In order to improve the image deraining quality, a multi-scale fusion self attention generation adversarial network (MSFSA-GAN) is proposed. This network uses different scales to e...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9741786/ |
| _version_ | 1811284730532855808 |
|---|---|
| author | Wang Xue Cheng Huan-Xin Sun Sheng-Yi Jiang Ze-Qin Cheng Kai Cheng Li |
| author_facet | Wang Xue Cheng Huan-Xin Sun Sheng-Yi Jiang Ze-Qin Cheng Kai Cheng Li |
| author_sort | Wang Xue |
| collection | DOAJ |
| description | Bad weather such as rainy days will seriously affect the image quality and the accuracy of visual processing algorithm. In order to improve the image deraining quality, a multi-scale fusion self attention generation adversarial network (MSFSA-GAN) is proposed. This network uses different scales to extract input characteristics of rain lines. First, Gaussian pyramid rain maps with different scales are generated by Gaussian algorithm. Then, in order to extract the features of rain lines with different scales, the coarse fusion module and fine fusion module are designed respectively. Next, the extracted features are fused at different scales. In this process, the self attention mechanism is introduced to make the network focus on the extracted features of different scales. And before the fusion, the rain pattern reconstruction operation is also carried out, so that the network can reproduce the input image more perfectly. Finally, it is input into the discriminator network with dense blocks to obtain the image that removes the rain lines. We used R100H and R100L datasets to train and test our network. The results show that our method as high as 27.79 in PSNR and UQI is 0.94, which is superior to the existing methods in performance. Meanwhile, we also compared the cost of time, the result of our network is only 0.02s. |
| first_indexed | 2024-04-13T02:33:25Z |
| format | Article |
| id | doaj.art-77001461c65c4ff1ba446a08c0195e96 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-13T02:33:25Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-77001461c65c4ff1ba446a08c0195e962022-12-22T03:06:28ZengIEEEIEEE Access2169-35362022-01-0110344423444810.1109/ACCESS.2022.31622249741786MSFSA-GAN: Multi-Scale Fusion Self Attention Generative Adversarial Network for Single Image DerainingWang Xue0https://orcid.org/0000-0003-1695-1292Cheng Huan-Xin1Sun Sheng-Yi2Jiang Ze-Qin3Cheng Kai4Cheng Li5School of Electronic Engineering and Automation, Qingdao, ChinaSchool of Electronic Engineering and Automation, Qingdao, ChinaSchool of Electronic Engineering and Automation, Qingdao, ChinaSchool of Electronic Engineering and Automation, Qingdao, ChinaSchool of Electronic Engineering and Automation, Qingdao, ChinaXinjiang Technical Institute of Physics & Chemistry, Urumqi, ChinaBad weather such as rainy days will seriously affect the image quality and the accuracy of visual processing algorithm. In order to improve the image deraining quality, a multi-scale fusion self attention generation adversarial network (MSFSA-GAN) is proposed. This network uses different scales to extract input characteristics of rain lines. First, Gaussian pyramid rain maps with different scales are generated by Gaussian algorithm. Then, in order to extract the features of rain lines with different scales, the coarse fusion module and fine fusion module are designed respectively. Next, the extracted features are fused at different scales. In this process, the self attention mechanism is introduced to make the network focus on the extracted features of different scales. And before the fusion, the rain pattern reconstruction operation is also carried out, so that the network can reproduce the input image more perfectly. Finally, it is input into the discriminator network with dense blocks to obtain the image that removes the rain lines. We used R100H and R100L datasets to train and test our network. The results show that our method as high as 27.79 in PSNR and UQI is 0.94, which is superior to the existing methods in performance. Meanwhile, we also compared the cost of time, the result of our network is only 0.02s.https://ieeexplore.ieee.org/document/9741786/Rain removalMSFSA-GANself attentiondense block |
| spellingShingle | Wang Xue Cheng Huan-Xin Sun Sheng-Yi Jiang Ze-Qin Cheng Kai Cheng Li MSFSA-GAN: Multi-Scale Fusion Self Attention Generative Adversarial Network for Single Image Deraining IEEE Access Rain removal MSFSA-GAN self attention dense block |
| title | MSFSA-GAN: Multi-Scale Fusion Self Attention Generative Adversarial Network for Single Image Deraining |
| title_full | MSFSA-GAN: Multi-Scale Fusion Self Attention Generative Adversarial Network for Single Image Deraining |
| title_fullStr | MSFSA-GAN: Multi-Scale Fusion Self Attention Generative Adversarial Network for Single Image Deraining |
| title_full_unstemmed | MSFSA-GAN: Multi-Scale Fusion Self Attention Generative Adversarial Network for Single Image Deraining |
| title_short | MSFSA-GAN: Multi-Scale Fusion Self Attention Generative Adversarial Network for Single Image Deraining |
| title_sort | msfsa gan multi scale fusion self attention generative adversarial network for single image deraining |
| topic | Rain removal MSFSA-GAN self attention dense block |
| url | https://ieeexplore.ieee.org/document/9741786/ |
| work_keys_str_mv | AT wangxue msfsaganmultiscalefusionselfattentiongenerativeadversarialnetworkforsingleimagederaining AT chenghuanxin msfsaganmultiscalefusionselfattentiongenerativeadversarialnetworkforsingleimagederaining AT sunshengyi msfsaganmultiscalefusionselfattentiongenerativeadversarialnetworkforsingleimagederaining AT jiangzeqin msfsaganmultiscalefusionselfattentiongenerativeadversarialnetworkforsingleimagederaining AT chengkai msfsaganmultiscalefusionselfattentiongenerativeadversarialnetworkforsingleimagederaining AT chengli msfsaganmultiscalefusionselfattentiongenerativeadversarialnetworkforsingleimagederaining |