Burst-Enhanced Super-Resolution Network (BESR)
Multi-frame super-resolution (MFSR) leverages complementary information between image sequences of the same scene to increase the resolution of the reconstructed image. As a branch of MFSR, burst super-resolution aims to restore image details by leveraging the complementary information between noisy...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2024-03-01
|
Series: | Sensors |
Subjects: | |
Online Access: | https://www.mdpi.com/1424-8220/24/7/2052 |
_version_ | 1797212037391056896 |
---|---|
author | Jiaao Li Qunbo Lv Wenjian Zhang Yu Zhang Zheng Tan |
author_facet | Jiaao Li Qunbo Lv Wenjian Zhang Yu Zhang Zheng Tan |
author_sort | Jiaao Li |
collection | DOAJ |
description | Multi-frame super-resolution (MFSR) leverages complementary information between image sequences of the same scene to increase the resolution of the reconstructed image. As a branch of MFSR, burst super-resolution aims to restore image details by leveraging the complementary information between noisy sequences. In this paper, we propose an efficient burst-enhanced super-resolution network (BESR). Specifically, we introduce Geformer, a gate-enhanced transformer, and construct an enhanced CNN-Transformer block (ECTB) by combining convolutions to enhance local perception. ECTB efficiently aggregates intra-frame context and inter-frame correlation information, yielding an enhanced feature representation. Additionally, we leverage reference features to facilitate inter-frame communication, enhancing spatiotemporal coherence among multiple frames. To address the critical processes of inter-frame alignment and feature fusion, we propose optimized pyramid alignment (OPA) and hybrid feature fusion (HFF) modules to capture and utilize complementary information between multiple frames to recover more high-frequency details. Extensive experiments demonstrate that, compared to state-of-the-art methods, BESR achieves higher efficiency and competitively superior reconstruction results. On the synthetic dataset and real-world dataset of BurstSR, our BESR achieves PSNR values of 42.79 dB and 48.86 dB, respectively, outperforming other MFSR models significantly. |
first_indexed | 2024-04-24T10:36:01Z |
format | Article |
id | doaj.art-b9eef6f0fb17426080dee0b89975aa3d |
institution | Directory Open Access Journal |
issn | 1424-8220 |
language | English |
last_indexed | 2024-04-24T10:36:01Z |
publishDate | 2024-03-01 |
publisher | MDPI AG |
record_format | Article |
series | Sensors |
spelling | doaj.art-b9eef6f0fb17426080dee0b89975aa3d2024-04-12T13:26:05ZengMDPI AGSensors1424-82202024-03-01247205210.3390/s24072052Burst-Enhanced Super-Resolution Network (BESR)Jiaao Li0Qunbo Lv1Wenjian Zhang2Yu Zhang3Zheng Tan4Aerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaMulti-frame super-resolution (MFSR) leverages complementary information between image sequences of the same scene to increase the resolution of the reconstructed image. As a branch of MFSR, burst super-resolution aims to restore image details by leveraging the complementary information between noisy sequences. In this paper, we propose an efficient burst-enhanced super-resolution network (BESR). Specifically, we introduce Geformer, a gate-enhanced transformer, and construct an enhanced CNN-Transformer block (ECTB) by combining convolutions to enhance local perception. ECTB efficiently aggregates intra-frame context and inter-frame correlation information, yielding an enhanced feature representation. Additionally, we leverage reference features to facilitate inter-frame communication, enhancing spatiotemporal coherence among multiple frames. To address the critical processes of inter-frame alignment and feature fusion, we propose optimized pyramid alignment (OPA) and hybrid feature fusion (HFF) modules to capture and utilize complementary information between multiple frames to recover more high-frequency details. Extensive experiments demonstrate that, compared to state-of-the-art methods, BESR achieves higher efficiency and competitively superior reconstruction results. On the synthetic dataset and real-world dataset of BurstSR, our BESR achieves PSNR values of 42.79 dB and 48.86 dB, respectively, outperforming other MFSR models significantly.https://www.mdpi.com/1424-8220/24/7/2052burst super-resolutionCNN-Transformermulti-frame super-resolution |
spellingShingle | Jiaao Li Qunbo Lv Wenjian Zhang Yu Zhang Zheng Tan Burst-Enhanced Super-Resolution Network (BESR) Sensors burst super-resolution CNN-Transformer multi-frame super-resolution |
title | Burst-Enhanced Super-Resolution Network (BESR) |
title_full | Burst-Enhanced Super-Resolution Network (BESR) |
title_fullStr | Burst-Enhanced Super-Resolution Network (BESR) |
title_full_unstemmed | Burst-Enhanced Super-Resolution Network (BESR) |
title_short | Burst-Enhanced Super-Resolution Network (BESR) |
title_sort | burst enhanced super resolution network besr |
topic | burst super-resolution CNN-Transformer multi-frame super-resolution |
url | https://www.mdpi.com/1424-8220/24/7/2052 |
work_keys_str_mv | AT jiaaoli burstenhancedsuperresolutionnetworkbesr AT qunbolv burstenhancedsuperresolutionnetworkbesr AT wenjianzhang burstenhancedsuperresolutionnetworkbesr AT yuzhang burstenhancedsuperresolutionnetworkbesr AT zhengtan burstenhancedsuperresolutionnetworkbesr |