TGLFusion: A Temperature-Guided Lightweight Fusion Method for Infrared and Visible Images
The fusion of infrared and visible images is a well-researched task in computer vision. These fusion methods create fused images replacing the manual observation of single sensor image, often deployed on edge devices for real-time processing. However, there is an issue of information imbalance betwe...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/6/1735 |
| _version_ | 1827305066368860160 |
|---|---|
| author | Bao Yan Longjie Zhao Kehua Miao Song Wang Qinghua Li Delin Luo |
| author_facet | Bao Yan Longjie Zhao Kehua Miao Song Wang Qinghua Li Delin Luo |
| author_sort | Bao Yan |
| collection | DOAJ |
| description | The fusion of infrared and visible images is a well-researched task in computer vision. These fusion methods create fused images replacing the manual observation of single sensor image, often deployed on edge devices for real-time processing. However, there is an issue of information imbalance between infrared and visible images. Existing methods often fail to emphasize temperature and edge texture information, potentially leading to misinterpretations. Moreover, these methods are computationally complex, and challenging for edge device adaptation. This paper proposes a method that calculates the distribution proportion of infrared pixel values, allocating fusion weights to adaptively highlight key information. It introduces a weight allocation mechanism and MobileBlock with a multispectral information complementary module, innovations which strengthened the model’s fusion capabilities, made it more lightweight, and ensured information compensation. Training involves a temperature-color-perception loss function, enabling adaptive weight allocation based on image pair information. Experimental results show superiority over mainstream fusion methods, particularly in the electric power equipment scene and publicly available datasets. |
| first_indexed | 2024-04-24T17:50:54Z |
| format | Article |
| id | doaj.art-b604cd4494e04d7eb844ce2d59a073e8 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-24T17:50:54Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-b604cd4494e04d7eb844ce2d59a073e82024-03-27T14:03:36ZengMDPI AGSensors1424-82202024-03-01246173510.3390/s24061735TGLFusion: A Temperature-Guided Lightweight Fusion Method for Infrared and Visible ImagesBao Yan0Longjie Zhao1Kehua Miao2Song Wang3Qinghua Li4Delin Luo5School of Aerospace Engineering, Xiamen University, Xiamen 361102, ChinaSchool of Aerospace Engineering, Xiamen University, Xiamen 361102, ChinaSchool of Aerospace Engineering, Xiamen University, Xiamen 361102, ChinaElectric Power Research Institute, China Southern Power Grid, Guangzhou 510063, ChinaSchool of Aerospace Engineering, Xiamen University, Xiamen 361102, ChinaSchool of Aerospace Engineering, Xiamen University, Xiamen 361102, ChinaThe fusion of infrared and visible images is a well-researched task in computer vision. These fusion methods create fused images replacing the manual observation of single sensor image, often deployed on edge devices for real-time processing. However, there is an issue of information imbalance between infrared and visible images. Existing methods often fail to emphasize temperature and edge texture information, potentially leading to misinterpretations. Moreover, these methods are computationally complex, and challenging for edge device adaptation. This paper proposes a method that calculates the distribution proportion of infrared pixel values, allocating fusion weights to adaptively highlight key information. It introduces a weight allocation mechanism and MobileBlock with a multispectral information complementary module, innovations which strengthened the model’s fusion capabilities, made it more lightweight, and ensured information compensation. Training involves a temperature-color-perception loss function, enabling adaptive weight allocation based on image pair information. Experimental results show superiority over mainstream fusion methods, particularly in the electric power equipment scene and publicly available datasets.https://www.mdpi.com/1424-8220/24/6/1735deep learningimage fusioninfrared and visible sensor imageslightweight modelelectric power equipment |
| spellingShingle | Bao Yan Longjie Zhao Kehua Miao Song Wang Qinghua Li Delin Luo TGLFusion: A Temperature-Guided Lightweight Fusion Method for Infrared and Visible Images Sensors deep learning image fusion infrared and visible sensor images lightweight model electric power equipment |
| title | TGLFusion: A Temperature-Guided Lightweight Fusion Method for Infrared and Visible Images |
| title_full | TGLFusion: A Temperature-Guided Lightweight Fusion Method for Infrared and Visible Images |
| title_fullStr | TGLFusion: A Temperature-Guided Lightweight Fusion Method for Infrared and Visible Images |
| title_full_unstemmed | TGLFusion: A Temperature-Guided Lightweight Fusion Method for Infrared and Visible Images |
| title_short | TGLFusion: A Temperature-Guided Lightweight Fusion Method for Infrared and Visible Images |
| title_sort | tglfusion a temperature guided lightweight fusion method for infrared and visible images |
| topic | deep learning image fusion infrared and visible sensor images lightweight model electric power equipment |
| url | https://www.mdpi.com/1424-8220/24/6/1735 |
| work_keys_str_mv | AT baoyan tglfusionatemperatureguidedlightweightfusionmethodforinfraredandvisibleimages AT longjiezhao tglfusionatemperatureguidedlightweightfusionmethodforinfraredandvisibleimages AT kehuamiao tglfusionatemperatureguidedlightweightfusionmethodforinfraredandvisibleimages AT songwang tglfusionatemperatureguidedlightweightfusionmethodforinfraredandvisibleimages AT qinghuali tglfusionatemperatureguidedlightweightfusionmethodforinfraredandvisibleimages AT delinluo tglfusionatemperatureguidedlightweightfusionmethodforinfraredandvisibleimages |