A Method of Aerial Multi-Modal Image Registration for a Low-Visibility Approach Based on Virtual Reality Fusion
Aiming at the approach and landing of an aircraft under low visibility, this paper studies the use of an infrared heat-transfer imaging camera and visible-light camera to obtain dynamic hyperspectral images of flight approach scenes from the perspective of enhancing pilot vision. Aiming at the probl...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/6/3396 |
| _version_ | 1797613756659793920 |
|---|---|
| author | Yuezhou Wu Changjiang Liu |
| author_facet | Yuezhou Wu Changjiang Liu |
| author_sort | Yuezhou Wu |
| collection | DOAJ |
| description | Aiming at the approach and landing of an aircraft under low visibility, this paper studies the use of an infrared heat-transfer imaging camera and visible-light camera to obtain dynamic hyperspectral images of flight approach scenes from the perspective of enhancing pilot vision. Aiming at the problems of affine deformation, difficulty in extracting similar geometric features, thermal shadows, light shadows, and other issues in heterogenous infrared and visible-light image registration, a multi-modal image registration method based on RoI driving in a virtual scene, RoI feature extraction, and virtual-reality-fusion-based contour angle orientation is proposed, and this could reduce the area to be registered, reduces the amount of computation, and improves the real-time registration accuracy. Aiming at the differences in multi-modal image fusion in terms of resolution, contrast, color channel, color information strength, and other aspects, the contour angle orientation maintains the geometric deformation of multi-source images well, and the virtual reality fusion technology effectively deletes incorrectly matched point pairs. By integrating redundant information and complementary information from multi-modal images, the visual perception abilities of pilots during the approach process are enhanced as a whole. |
| first_indexed | 2024-03-11T07:00:07Z |
| format | Article |
| id | doaj.art-ff4603eef33f47e89d3a7f89b856a4a5 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T07:00:07Z |
| publishDate | 2023-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-ff4603eef33f47e89d3a7f89b856a4a52023-11-17T09:21:05ZengMDPI AGApplied Sciences2076-34172023-03-01136339610.3390/app13063396A Method of Aerial Multi-Modal Image Registration for a Low-Visibility Approach Based on Virtual Reality FusionYuezhou Wu0Changjiang Liu1School of Computer Science, Civil Aviation Flight University of China, Guanghan 618307, ChinaKey Laboratory of Higher Education of Sichuan Province for Enterprise Informationalization and Internet of Things, Sichuan University of Sciencce and Engineering, Zigong 643000, ChinaAiming at the approach and landing of an aircraft under low visibility, this paper studies the use of an infrared heat-transfer imaging camera and visible-light camera to obtain dynamic hyperspectral images of flight approach scenes from the perspective of enhancing pilot vision. Aiming at the problems of affine deformation, difficulty in extracting similar geometric features, thermal shadows, light shadows, and other issues in heterogenous infrared and visible-light image registration, a multi-modal image registration method based on RoI driving in a virtual scene, RoI feature extraction, and virtual-reality-fusion-based contour angle orientation is proposed, and this could reduce the area to be registered, reduces the amount of computation, and improves the real-time registration accuracy. Aiming at the differences in multi-modal image fusion in terms of resolution, contrast, color channel, color information strength, and other aspects, the contour angle orientation maintains the geometric deformation of multi-source images well, and the virtual reality fusion technology effectively deletes incorrectly matched point pairs. By integrating redundant information and complementary information from multi-modal images, the visual perception abilities of pilots during the approach process are enhanced as a whole.https://www.mdpi.com/2076-3417/13/6/3396infrared imagemulti-modal imagesimage registrationimage fusionEVS (enhanced vision system) |
| spellingShingle | Yuezhou Wu Changjiang Liu A Method of Aerial Multi-Modal Image Registration for a Low-Visibility Approach Based on Virtual Reality Fusion Applied Sciences infrared image multi-modal images image registration image fusion EVS (enhanced vision system) |
| title | A Method of Aerial Multi-Modal Image Registration for a Low-Visibility Approach Based on Virtual Reality Fusion |
| title_full | A Method of Aerial Multi-Modal Image Registration for a Low-Visibility Approach Based on Virtual Reality Fusion |
| title_fullStr | A Method of Aerial Multi-Modal Image Registration for a Low-Visibility Approach Based on Virtual Reality Fusion |
| title_full_unstemmed | A Method of Aerial Multi-Modal Image Registration for a Low-Visibility Approach Based on Virtual Reality Fusion |
| title_short | A Method of Aerial Multi-Modal Image Registration for a Low-Visibility Approach Based on Virtual Reality Fusion |
| title_sort | method of aerial multi modal image registration for a low visibility approach based on virtual reality fusion |
| topic | infrared image multi-modal images image registration image fusion EVS (enhanced vision system) |
| url | https://www.mdpi.com/2076-3417/13/6/3396 |
| work_keys_str_mv | AT yuezhouwu amethodofaerialmultimodalimageregistrationforalowvisibilityapproachbasedonvirtualrealityfusion AT changjiangliu amethodofaerialmultimodalimageregistrationforalowvisibilityapproachbasedonvirtualrealityfusion AT yuezhouwu methodofaerialmultimodalimageregistrationforalowvisibilityapproachbasedonvirtualrealityfusion AT changjiangliu methodofaerialmultimodalimageregistrationforalowvisibilityapproachbasedonvirtualrealityfusion |