MCNN: Conditional focus probability learning to multi‐focus image fusion via mutually coupled neural network
Abstract In this paper, a novel conditional focus probability learning model, termed MCNN, is proposed for multi‐focus image fusion (MFIF). Given a pair of source images, their conditional focus probabilities can be generated by using the well‐trained MCNN, which is further converted into the binary...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-06-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12805 |
| _version_ | 1797813837792018432 |
|---|---|
| author | Chengchao Wang Yuanyuan Pu Xue Wang Chaozhen Ma Rencan Nie |
| author_facet | Chengchao Wang Yuanyuan Pu Xue Wang Chaozhen Ma Rencan Nie |
| author_sort | Chengchao Wang |
| collection | DOAJ |
| description | Abstract In this paper, a novel conditional focus probability learning model, termed MCNN, is proposed for multi‐focus image fusion (MFIF). Given a pair of source images, their conditional focus probabilities can be generated by using the well‐trained MCNN, which is further converted into the binary focus masks to directly produce an all‐focus image with no postprocessing. To this end, a fully convolutional encoder is designed with two mutually coupled Siamese branches in MCNN, which include a coupling block that bridge between the two branches to provide conditional information to each other, at different layers, such that the encoder can more strongly extract conditional focus features and further encourage the decoder pixel‐wisely to give more robust conditional focus probabilities. Moreover, a hybrid loss is designed with a structural sparse fidelity loss and a structural similarity loss to force the network to learn more accurate conditional focus probabilities. Particularly, a convolutional norm with good structural group sparse is proposed, to construct the structural sparse fidelity loss. Simulation results substantiate the superiority of our MCNN over other state‐of‐the‐art, in terms of both visual perception and quantitative evaluation. |
| first_indexed | 2024-03-13T07:58:36Z |
| format | Article |
| id | doaj.art-ef93ae10db27427fb0db2fd29ca77422 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-03-13T07:58:36Z |
| publishDate | 2023-06-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-ef93ae10db27427fb0db2fd29ca774222023-06-02T03:06:38ZengWileyIET Image Processing1751-96591751-96672023-06-011782422243610.1049/ipr2.12805MCNN: Conditional focus probability learning to multi‐focus image fusion via mutually coupled neural networkChengchao Wang0Yuanyuan Pu1Xue Wang2Chaozhen Ma3Rencan Nie4School of Information Science and Engineering Yunnan University Kunming ChinaSchool of Information Science and Engineering Yunnan University Kunming ChinaSchool of Information Science and Engineering Yunnan University Kunming ChinaSchool of Information Science and Engineering Yunnan University Kunming ChinaSchool of Information Science and Engineering Yunnan University Kunming ChinaAbstract In this paper, a novel conditional focus probability learning model, termed MCNN, is proposed for multi‐focus image fusion (MFIF). Given a pair of source images, their conditional focus probabilities can be generated by using the well‐trained MCNN, which is further converted into the binary focus masks to directly produce an all‐focus image with no postprocessing. To this end, a fully convolutional encoder is designed with two mutually coupled Siamese branches in MCNN, which include a coupling block that bridge between the two branches to provide conditional information to each other, at different layers, such that the encoder can more strongly extract conditional focus features and further encourage the decoder pixel‐wisely to give more robust conditional focus probabilities. Moreover, a hybrid loss is designed with a structural sparse fidelity loss and a structural similarity loss to force the network to learn more accurate conditional focus probabilities. Particularly, a convolutional norm with good structural group sparse is proposed, to construct the structural sparse fidelity loss. Simulation results substantiate the superiority of our MCNN over other state‐of‐the‐art, in terms of both visual perception and quantitative evaluation.https://doi.org/10.1049/ipr2.12805Multi‐focus image fusionconditional focus probabilityconvolutional neural networkhybrid loss |
| spellingShingle | Chengchao Wang Yuanyuan Pu Xue Wang Chaozhen Ma Rencan Nie MCNN: Conditional focus probability learning to multi‐focus image fusion via mutually coupled neural network IET Image Processing Multi‐focus image fusion conditional focus probability convolutional neural network hybrid loss |
| title | MCNN: Conditional focus probability learning to multi‐focus image fusion via mutually coupled neural network |
| title_full | MCNN: Conditional focus probability learning to multi‐focus image fusion via mutually coupled neural network |
| title_fullStr | MCNN: Conditional focus probability learning to multi‐focus image fusion via mutually coupled neural network |
| title_full_unstemmed | MCNN: Conditional focus probability learning to multi‐focus image fusion via mutually coupled neural network |
| title_short | MCNN: Conditional focus probability learning to multi‐focus image fusion via mutually coupled neural network |
| title_sort | mcnn conditional focus probability learning to multi focus image fusion via mutually coupled neural network |
| topic | Multi‐focus image fusion conditional focus probability convolutional neural network hybrid loss |
| url | https://doi.org/10.1049/ipr2.12805 |
| work_keys_str_mv | AT chengchaowang mcnnconditionalfocusprobabilitylearningtomultifocusimagefusionviamutuallycoupledneuralnetwork AT yuanyuanpu mcnnconditionalfocusprobabilitylearningtomultifocusimagefusionviamutuallycoupledneuralnetwork AT xuewang mcnnconditionalfocusprobabilitylearningtomultifocusimagefusionviamutuallycoupledneuralnetwork AT chaozhenma mcnnconditionalfocusprobabilitylearningtomultifocusimagefusionviamutuallycoupledneuralnetwork AT rencannie mcnnconditionalfocusprobabilitylearningtomultifocusimagefusionviamutuallycoupledneuralnetwork |