Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching
As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image sti...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/17/7488 |
| _version_ | 1797581879900110848 |
|---|---|
| author | Xiaoting Fan Long Sun Zhong Zhang Shuang Liu Tariq S. Durrani |
| author_facet | Xiaoting Fan Long Sun Zhong Zhang Shuang Liu Tariq S. Durrani |
| author_sort | Xiaoting Fan |
| collection | DOAJ |
| description | As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image stitching only conduct a single deep homography to perform image alignment, which may produce inevitable alignment distortions. To address this issue, we propose a content-seam-preserving multi-alignment network (CSPM-Net) for visual-sensor-based image stitching, which could preserve the image content consistency and avoid seam distortions simultaneously. Firstly, a content-preserving deep homography estimation was designed to pre-align the input image pairs and reduce the content inconsistency. Secondly, an edge-assisted mesh warping was conducted to further align the image pairs, where the edge information is introduced to eliminate seam artifacts. Finally, in order to predict the final stitched image accurately, a content consistency loss was designed to preserve the geometric structure of overlapping regions between image pairs, and a seam smoothness loss is proposed to eliminate the edge distortions of image boundaries. Experimental results demonstrated that the proposed image-stitching method can provide favorable stitching results for visual-sensor-based images and outperform other state-of-the-art methods. |
| first_indexed | 2024-03-10T23:13:54Z |
| format | Article |
| id | doaj.art-e83e2de709fc46f687136914f71bb372 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T23:13:54Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-e83e2de709fc46f687136914f71bb3722023-11-19T08:50:33ZengMDPI AGSensors1424-82202023-08-012317748810.3390/s23177488Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image StitchingXiaoting Fan0Long Sun1Zhong Zhang2Shuang Liu3Tariq S. Durrani4Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin 300387, ChinaSchool of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaTianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin 300387, ChinaTianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin 300387, ChinaDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XQ, UKAs an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image stitching only conduct a single deep homography to perform image alignment, which may produce inevitable alignment distortions. To address this issue, we propose a content-seam-preserving multi-alignment network (CSPM-Net) for visual-sensor-based image stitching, which could preserve the image content consistency and avoid seam distortions simultaneously. Firstly, a content-preserving deep homography estimation was designed to pre-align the input image pairs and reduce the content inconsistency. Secondly, an edge-assisted mesh warping was conducted to further align the image pairs, where the edge information is introduced to eliminate seam artifacts. Finally, in order to predict the final stitched image accurately, a content consistency loss was designed to preserve the geometric structure of overlapping regions between image pairs, and a seam smoothness loss is proposed to eliminate the edge distortions of image boundaries. Experimental results demonstrated that the proposed image-stitching method can provide favorable stitching results for visual-sensor-based images and outperform other state-of-the-art methods.https://www.mdpi.com/1424-8220/23/17/7488visual-sensor-based image stitchingdeep homographymesh warpingcontent-preservingedge-assisted |
| spellingShingle | Xiaoting Fan Long Sun Zhong Zhang Shuang Liu Tariq S. Durrani Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching Sensors visual-sensor-based image stitching deep homography mesh warping content-preserving edge-assisted |
| title | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
| title_full | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
| title_fullStr | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
| title_full_unstemmed | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
| title_short | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
| title_sort | content seam preserving multi alignment network for visual sensor based image stitching |
| topic | visual-sensor-based image stitching deep homography mesh warping content-preserving edge-assisted |
| url | https://www.mdpi.com/1424-8220/23/17/7488 |
| work_keys_str_mv | AT xiaotingfan contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching AT longsun contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching AT zhongzhang contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching AT shuangliu contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching AT tariqsdurrani contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching |