A multimodal feature fusion image dehazing method with scene depth prior
Abstract Current dehazing networks usually only learn haze features in a single‐image colour space and often suffer from uneven dehazing, colour, and edge degradation when confronted with different scales of ground objects in the depth space of the scene. The authors propose a multimodal feature fus...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-09-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12866 |
| _version_ | 1797693053613375488 |
|---|---|
| author | Zhang Zhengpeng Cheng Yan Zhang Shuai Bu Lijing Deng Mingjun |
| author_facet | Zhang Zhengpeng Cheng Yan Zhang Shuai Bu Lijing Deng Mingjun |
| author_sort | Zhang Zhengpeng |
| collection | DOAJ |
| description | Abstract Current dehazing networks usually only learn haze features in a single‐image colour space and often suffer from uneven dehazing, colour, and edge degradation when confronted with different scales of ground objects in the depth space of the scene. The authors propose a multimodal feature fusion image dehazing method with scene depth prior based on a decoder–encoder backbone network. The multimodal feature fusion module was first designed. In this module, affine transformation and polarized self‐attention mechanism are used to realize the fusion of image colour and depth prior feature, to improve the representation ability of the model for different scale ground haze feature in‐depth space. Then, the feature enhancement module (FEM) is added, and deformable convolution and difference convolution methods are used to enhance the representation ability of the model for the geometric and texture feature of the ground objects. The publicly available dehazing datasets are used for comparison and ablation experiments. The results show that compared with the existing classical dehazing networks, the peak signal‐to‐noise ratio (PSNR) and SSIM of the authors’ proposed method have been significantly improved, have a more uniform dehazing effect in different depth spaces, and maintain the colour and edge details of the ground objects very well. |
| first_indexed | 2024-03-12T02:38:01Z |
| format | Article |
| id | doaj.art-18bf0ea96ed14dcaadc47ee117a917c6 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-03-12T02:38:01Z |
| publishDate | 2023-09-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-18bf0ea96ed14dcaadc47ee117a917c62023-09-04T10:54:49ZengWileyIET Image Processing1751-96591751-96672023-09-0117113079309410.1049/ipr2.12866A multimodal feature fusion image dehazing method with scene depth priorZhang Zhengpeng0Cheng Yan1Zhang Shuai2Bu Lijing3Deng Mingjun4School of Automation and Electronic Information Xiangtan University XiangtanChinaSchool of Automation and Electronic Information Xiangtan University XiangtanChinaSchool of Geomatics Liaoning Technical University FuxinLiaoningChinaSchool of Automation and Electronic Information Xiangtan University XiangtanChinaSchool of Automation and Electronic Information Xiangtan University XiangtanChinaAbstract Current dehazing networks usually only learn haze features in a single‐image colour space and often suffer from uneven dehazing, colour, and edge degradation when confronted with different scales of ground objects in the depth space of the scene. The authors propose a multimodal feature fusion image dehazing method with scene depth prior based on a decoder–encoder backbone network. The multimodal feature fusion module was first designed. In this module, affine transformation and polarized self‐attention mechanism are used to realize the fusion of image colour and depth prior feature, to improve the representation ability of the model for different scale ground haze feature in‐depth space. Then, the feature enhancement module (FEM) is added, and deformable convolution and difference convolution methods are used to enhance the representation ability of the model for the geometric and texture feature of the ground objects. The publicly available dehazing datasets are used for comparison and ablation experiments. The results show that compared with the existing classical dehazing networks, the peak signal‐to‐noise ratio (PSNR) and SSIM of the authors’ proposed method have been significantly improved, have a more uniform dehazing effect in different depth spaces, and maintain the colour and edge details of the ground objects very well.https://doi.org/10.1049/ipr2.12866attention mechanismsdeep priordehazing networkdifference convolutionmultimodal feature fusion |
| spellingShingle | Zhang Zhengpeng Cheng Yan Zhang Shuai Bu Lijing Deng Mingjun A multimodal feature fusion image dehazing method with scene depth prior IET Image Processing attention mechanisms deep prior dehazing network difference convolution multimodal feature fusion |
| title | A multimodal feature fusion image dehazing method with scene depth prior |
| title_full | A multimodal feature fusion image dehazing method with scene depth prior |
| title_fullStr | A multimodal feature fusion image dehazing method with scene depth prior |
| title_full_unstemmed | A multimodal feature fusion image dehazing method with scene depth prior |
| title_short | A multimodal feature fusion image dehazing method with scene depth prior |
| title_sort | multimodal feature fusion image dehazing method with scene depth prior |
| topic | attention mechanisms deep prior dehazing network difference convolution multimodal feature fusion |
| url | https://doi.org/10.1049/ipr2.12866 |
| work_keys_str_mv | AT zhangzhengpeng amultimodalfeaturefusionimagedehazingmethodwithscenedepthprior AT chengyan amultimodalfeaturefusionimagedehazingmethodwithscenedepthprior AT zhangshuai amultimodalfeaturefusionimagedehazingmethodwithscenedepthprior AT bulijing amultimodalfeaturefusionimagedehazingmethodwithscenedepthprior AT dengmingjun amultimodalfeaturefusionimagedehazingmethodwithscenedepthprior AT zhangzhengpeng multimodalfeaturefusionimagedehazingmethodwithscenedepthprior AT chengyan multimodalfeaturefusionimagedehazingmethodwithscenedepthprior AT zhangshuai multimodalfeaturefusionimagedehazingmethodwithscenedepthprior AT bulijing multimodalfeaturefusionimagedehazingmethodwithscenedepthprior AT dengmingjun multimodalfeaturefusionimagedehazingmethodwithscenedepthprior |