Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image
Removing raindrops from a single image is a challenging problem due to the complex changes in shape, scale, and transparency among raindrops. Previous explorations have mainly been limited in two ways. First, publicly available raindrop image datasets have limited capacity in terms of modeling raind...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/23/6733 |
| _version_ | 1797546840880578560 |
|---|---|
| author | Hao Luo Qingbo Wu King Ngi Ngan Hanxiao Luo Haoran Wei Hongliang Li Fanman Meng Linfeng Xu |
| author_facet | Hao Luo Qingbo Wu King Ngi Ngan Hanxiao Luo Haoran Wei Hongliang Li Fanman Meng Linfeng Xu |
| author_sort | Hao Luo |
| collection | DOAJ |
| description | Removing raindrops from a single image is a challenging problem due to the complex changes in shape, scale, and transparency among raindrops. Previous explorations have mainly been limited in two ways. First, publicly available raindrop image datasets have limited capacity in terms of modeling raindrop characteristics (e.g., raindrop collision and fusion) in real-world scenes. Second, recent deraining methods tend to apply shape-invariant filters to cope with diverse rainy images and fail to remove raindrops that are especially varied in shape and scale. In this paper, we address these raindrop removal problems from two perspectives. First, we establish a large-scale dataset named RaindropCityscapes, which includes 11,583 pairs of raindrop and raindrop-free images, covering a wide variety of raindrops and background scenarios. Second, a two-branch Multi-scale Shape Adaptive Network (MSANet) is proposed to detect and remove diverse raindrops, effectively filtering the occluded raindrop regions and keeping the clean background well-preserved. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art raindrop removal methods. Moreover, the extension of our method towards the rainy image segmentation and detection tasks validates the practicality of the proposed method in outdoor applications. |
| first_indexed | 2024-03-10T14:34:55Z |
| format | Article |
| id | doaj.art-5cfcc337d3964f4ca47629328e0182dc |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T14:34:55Z |
| publishDate | 2020-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-5cfcc337d3964f4ca47629328e0182dc2023-11-20T22:15:06ZengMDPI AGSensors1424-82202020-11-012023673310.3390/s20236733Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single ImageHao Luo0Qingbo Wu1King Ngi Ngan2Hanxiao Luo3Haoran Wei4Hongliang Li5Fanman Meng6Linfeng Xu7School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaRemoving raindrops from a single image is a challenging problem due to the complex changes in shape, scale, and transparency among raindrops. Previous explorations have mainly been limited in two ways. First, publicly available raindrop image datasets have limited capacity in terms of modeling raindrop characteristics (e.g., raindrop collision and fusion) in real-world scenes. Second, recent deraining methods tend to apply shape-invariant filters to cope with diverse rainy images and fail to remove raindrops that are especially varied in shape and scale. In this paper, we address these raindrop removal problems from two perspectives. First, we establish a large-scale dataset named RaindropCityscapes, which includes 11,583 pairs of raindrop and raindrop-free images, covering a wide variety of raindrops and background scenarios. Second, a two-branch Multi-scale Shape Adaptive Network (MSANet) is proposed to detect and remove diverse raindrops, effectively filtering the occluded raindrop regions and keeping the clean background well-preserved. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art raindrop removal methods. Moreover, the extension of our method towards the rainy image segmentation and detection tasks validates the practicality of the proposed method in outdoor applications.https://www.mdpi.com/1424-8220/20/23/6733shape adaptive networkraindrop and raindrop-free imagesraindrop detection and removaloccluded region filteringclean background preservation |
| spellingShingle | Hao Luo Qingbo Wu King Ngi Ngan Hanxiao Luo Haoran Wei Hongliang Li Fanman Meng Linfeng Xu Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image Sensors shape adaptive network raindrop and raindrop-free images raindrop detection and removal occluded region filtering clean background preservation |
| title | Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image |
| title_full | Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image |
| title_fullStr | Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image |
| title_full_unstemmed | Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image |
| title_short | Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image |
| title_sort | multi scale shape adaptive network for raindrop detection and removal from a single image |
| topic | shape adaptive network raindrop and raindrop-free images raindrop detection and removal occluded region filtering clean background preservation |
| url | https://www.mdpi.com/1424-8220/20/23/6733 |
| work_keys_str_mv | AT haoluo multiscaleshapeadaptivenetworkforraindropdetectionandremovalfromasingleimage AT qingbowu multiscaleshapeadaptivenetworkforraindropdetectionandremovalfromasingleimage AT kingngingan multiscaleshapeadaptivenetworkforraindropdetectionandremovalfromasingleimage AT hanxiaoluo multiscaleshapeadaptivenetworkforraindropdetectionandremovalfromasingleimage AT haoranwei multiscaleshapeadaptivenetworkforraindropdetectionandremovalfromasingleimage AT hongliangli multiscaleshapeadaptivenetworkforraindropdetectionandremovalfromasingleimage AT fanmanmeng multiscaleshapeadaptivenetworkforraindropdetectionandremovalfromasingleimage AT linfengxu multiscaleshapeadaptivenetworkforraindropdetectionandremovalfromasingleimage |