Observing Individuals and Behavior of Hainan Gibbons (<i>Nomascus hainanus</i>) Using Drone Infrared and Visible Image Fusion Technology
The Hainan gibbon (<i>Nomascus hainanus</i>) is one of the most endangered primates in the world. Infrared and visible images taken by drones are an important and effective way to observe Hainan gibbons. However, a single infrared or visible image cannot simultaneously observe the moveme...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Drones |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-446X/7/9/543 |
| _version_ | 1797580545071251456 |
|---|---|
| author | Shengshi Li Guanjun Wang Hui Zhang Yonghua Zou |
| author_facet | Shengshi Li Guanjun Wang Hui Zhang Yonghua Zou |
| author_sort | Shengshi Li |
| collection | DOAJ |
| description | The Hainan gibbon (<i>Nomascus hainanus</i>) is one of the most endangered primates in the world. Infrared and visible images taken by drones are an important and effective way to observe Hainan gibbons. However, a single infrared or visible image cannot simultaneously observe the movement tracks of Hainan gibbons and the appearance of the rainforest. The fusion of infrared and visible images of the same scene aims to generate a composite image which can provide a more comprehensive description of the scene. We propose a fusion method of infrared and visible images of the Hainan gibbon for the first time, termed Swin-UetFuse. The Swin-UetFuse has a powerful global and long-range semantic information extraction capability, which is very suitable for application in complex tropical rainforest environments. Firstly, the hierarchical Swin Transformer is applied as the encoder to extract the features of different scales of infrared and visible images. Secondly, the features of different scales are fused through the <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>l</mi><mn>1</mn></msub></semantics></math></inline-formula>-norm strategy. Finally, the Swing Transformer blocks and patch-expanding layers are utilized as the decoder to up-sample the fusion features to obtain the fused image. We used 21 pairs of Hainan gibbon datasets to perform experiments, and the experimental results demonstrate that the proposed method achieves excellent fusion performance. The infrared and visible image fusion technology of drones provides an important reference for the observation and protection of the Hainan gibbons. |
| first_indexed | 2024-03-10T22:52:31Z |
| format | Article |
| id | doaj.art-cb389a60dba546f28e7599e4750be751 |
| institution | Directory Open Access Journal |
| issn | 2504-446X |
| language | English |
| last_indexed | 2024-03-10T22:52:31Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Drones |
| spelling | doaj.art-cb389a60dba546f28e7599e4750be7512023-11-19T10:16:47ZengMDPI AGDrones2504-446X2023-08-017954310.3390/drones7090543Observing Individuals and Behavior of Hainan Gibbons (<i>Nomascus hainanus</i>) Using Drone Infrared and Visible Image Fusion TechnologyShengshi Li0Guanjun Wang1Hui Zhang2Yonghua Zou3School of Information and Communication Engineering, Hainan University, Haikou 570228, ChinaSchool of Information and Communication Engineering, Hainan University, Haikou 570228, ChinaKey Laboratory of Genetics and Germplasm Innovation of Tropical Special Forest Trees and Ornamental Plants, Ministry of Education, School of Forestry, Hainan University, Haikou 570228, ChinaSchool of Information and Communication Engineering, Hainan University, Haikou 570228, ChinaThe Hainan gibbon (<i>Nomascus hainanus</i>) is one of the most endangered primates in the world. Infrared and visible images taken by drones are an important and effective way to observe Hainan gibbons. However, a single infrared or visible image cannot simultaneously observe the movement tracks of Hainan gibbons and the appearance of the rainforest. The fusion of infrared and visible images of the same scene aims to generate a composite image which can provide a more comprehensive description of the scene. We propose a fusion method of infrared and visible images of the Hainan gibbon for the first time, termed Swin-UetFuse. The Swin-UetFuse has a powerful global and long-range semantic information extraction capability, which is very suitable for application in complex tropical rainforest environments. Firstly, the hierarchical Swin Transformer is applied as the encoder to extract the features of different scales of infrared and visible images. Secondly, the features of different scales are fused through the <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>l</mi><mn>1</mn></msub></semantics></math></inline-formula>-norm strategy. Finally, the Swing Transformer blocks and patch-expanding layers are utilized as the decoder to up-sample the fusion features to obtain the fused image. We used 21 pairs of Hainan gibbon datasets to perform experiments, and the experimental results demonstrate that the proposed method achieves excellent fusion performance. The infrared and visible image fusion technology of drones provides an important reference for the observation and protection of the Hainan gibbons.https://www.mdpi.com/2504-446X/7/9/543observation methodsUAVimage fusionHainan gibbonswin transformerskip connection |
| spellingShingle | Shengshi Li Guanjun Wang Hui Zhang Yonghua Zou Observing Individuals and Behavior of Hainan Gibbons (<i>Nomascus hainanus</i>) Using Drone Infrared and Visible Image Fusion Technology Drones observation methods UAV image fusion Hainan gibbon swin transformer skip connection |
| title | Observing Individuals and Behavior of Hainan Gibbons (<i>Nomascus hainanus</i>) Using Drone Infrared and Visible Image Fusion Technology |
| title_full | Observing Individuals and Behavior of Hainan Gibbons (<i>Nomascus hainanus</i>) Using Drone Infrared and Visible Image Fusion Technology |
| title_fullStr | Observing Individuals and Behavior of Hainan Gibbons (<i>Nomascus hainanus</i>) Using Drone Infrared and Visible Image Fusion Technology |
| title_full_unstemmed | Observing Individuals and Behavior of Hainan Gibbons (<i>Nomascus hainanus</i>) Using Drone Infrared and Visible Image Fusion Technology |
| title_short | Observing Individuals and Behavior of Hainan Gibbons (<i>Nomascus hainanus</i>) Using Drone Infrared and Visible Image Fusion Technology |
| title_sort | observing individuals and behavior of hainan gibbons i nomascus hainanus i using drone infrared and visible image fusion technology |
| topic | observation methods UAV image fusion Hainan gibbon swin transformer skip connection |
| url | https://www.mdpi.com/2504-446X/7/9/543 |
| work_keys_str_mv | AT shengshili observingindividualsandbehaviorofhainangibbonsinomascushainanusiusingdroneinfraredandvisibleimagefusiontechnology AT guanjunwang observingindividualsandbehaviorofhainangibbonsinomascushainanusiusingdroneinfraredandvisibleimagefusiontechnology AT huizhang observingindividualsandbehaviorofhainangibbonsinomascushainanusiusingdroneinfraredandvisibleimagefusiontechnology AT yonghuazou observingindividualsandbehaviorofhainangibbonsinomascushainanusiusingdroneinfraredandvisibleimagefusiontechnology |