Exhaustive Search of Correspondences between Multimodal Remote Sensing Images Using Convolutional Neural Network
Finding putative correspondences between a pair of images is an important prerequisite for image registration. In complex cases such as multimodal registration, a true match could be less plausible than a false match within a search zone. Under these conditions, it is important to detect all plausib...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/3/1231 |
| _version_ | 1797484541549477888 |
|---|---|
| author | Mykhail Uss Benoit Vozel Vladimir Lukin Kacem Chehdi |
| author_facet | Mykhail Uss Benoit Vozel Vladimir Lukin Kacem Chehdi |
| author_sort | Mykhail Uss |
| collection | DOAJ |
| description | Finding putative correspondences between a pair of images is an important prerequisite for image registration. In complex cases such as multimodal registration, a true match could be less plausible than a false match within a search zone. Under these conditions, it is important to detect all plausible matches. This could be achieved by an exhaustive search using a handcrafted similarity measure (SM, e.g., mutual information). It is promising to replace handcrafted SMs with deep learning ones that offer better performance. However, the latter are not designed for an exhaustive search of all matches but for finding the most plausible one. In this paper, we propose a deep-learning-based solution for exhaustive multiple match search between two images within a predefined search area. We design a computationally efficient convolutional neural network (CNN) that takes as input a template fragment from one image, a search fragment from another image and produces an SM map covering the entire search area in spatial dimensions. This SM map finds multiple plausible matches, locates each match with subpixel accuracy and provides a covariance matrix of localization errors for each match. The proposed CNN is trained with a specially designed loss function that enforces the translation and rotation invariance of the SM map and enables the detection of matches that have no associated ground truth data (e.g., multiple matches for repetitive textures). We validate the approach on multimodal remote sensing images and show that the proposed “area” SM performs better than “point” SM. |
| first_indexed | 2024-03-09T23:06:52Z |
| format | Article |
| id | doaj.art-61e26e2dae634737bcb75ca2821c0413 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T23:06:52Z |
| publishDate | 2022-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-61e26e2dae634737bcb75ca2821c04132023-11-23T17:52:38ZengMDPI AGSensors1424-82202022-02-01223123110.3390/s22031231Exhaustive Search of Correspondences between Multimodal Remote Sensing Images Using Convolutional Neural NetworkMykhail Uss0Benoit Vozel1Vladimir Lukin2Kacem Chehdi3Department of Information-Communication Technologies, National Aerospace University, Kharkov 61070, UkraineInstitut d’Electronique et des Technologies du numéRique, IETR UMR CNRS 6164, University of Rennes 1, 22305 Lannion, FranceDepartment of Information-Communication Technologies, National Aerospace University, Kharkov 61070, UkraineInstitut d’Electronique et des Technologies du numéRique, IETR UMR CNRS 6164, University of Rennes 1, 22305 Lannion, FranceFinding putative correspondences between a pair of images is an important prerequisite for image registration. In complex cases such as multimodal registration, a true match could be less plausible than a false match within a search zone. Under these conditions, it is important to detect all plausible matches. This could be achieved by an exhaustive search using a handcrafted similarity measure (SM, e.g., mutual information). It is promising to replace handcrafted SMs with deep learning ones that offer better performance. However, the latter are not designed for an exhaustive search of all matches but for finding the most plausible one. In this paper, we propose a deep-learning-based solution for exhaustive multiple match search between two images within a predefined search area. We design a computationally efficient convolutional neural network (CNN) that takes as input a template fragment from one image, a search fragment from another image and produces an SM map covering the entire search area in spatial dimensions. This SM map finds multiple plausible matches, locates each match with subpixel accuracy and provides a covariance matrix of localization errors for each match. The proposed CNN is trained with a specially designed loss function that enforces the translation and rotation invariance of the SM map and enables the detection of matches that have no associated ground truth data (e.g., multiple matches for repetitive textures). We validate the approach on multimodal remote sensing images and show that the proposed “area” SM performs better than “point” SM.https://www.mdpi.com/1424-8220/22/3/1231similarity measuremultimodal imagesexhaustive searchdeep learningmultiple correspondences |
| spellingShingle | Mykhail Uss Benoit Vozel Vladimir Lukin Kacem Chehdi Exhaustive Search of Correspondences between Multimodal Remote Sensing Images Using Convolutional Neural Network Sensors similarity measure multimodal images exhaustive search deep learning multiple correspondences |
| title | Exhaustive Search of Correspondences between Multimodal Remote Sensing Images Using Convolutional Neural Network |
| title_full | Exhaustive Search of Correspondences between Multimodal Remote Sensing Images Using Convolutional Neural Network |
| title_fullStr | Exhaustive Search of Correspondences between Multimodal Remote Sensing Images Using Convolutional Neural Network |
| title_full_unstemmed | Exhaustive Search of Correspondences between Multimodal Remote Sensing Images Using Convolutional Neural Network |
| title_short | Exhaustive Search of Correspondences between Multimodal Remote Sensing Images Using Convolutional Neural Network |
| title_sort | exhaustive search of correspondences between multimodal remote sensing images using convolutional neural network |
| topic | similarity measure multimodal images exhaustive search deep learning multiple correspondences |
| url | https://www.mdpi.com/1424-8220/22/3/1231 |
| work_keys_str_mv | AT mykhailuss exhaustivesearchofcorrespondencesbetweenmultimodalremotesensingimagesusingconvolutionalneuralnetwork AT benoitvozel exhaustivesearchofcorrespondencesbetweenmultimodalremotesensingimagesusingconvolutionalneuralnetwork AT vladimirlukin exhaustivesearchofcorrespondencesbetweenmultimodalremotesensingimagesusingconvolutionalneuralnetwork AT kacemchehdi exhaustivesearchofcorrespondencesbetweenmultimodalremotesensingimagesusingconvolutionalneuralnetwork |