Burst-Enhanced Super-Resolution Network (BESR)

Multi-frame super-resolution (MFSR) leverages complementary information between image sequences of the same scene to increase the resolution of the reconstructed image. As a branch of MFSR, burst super-resolution aims to restore image details by leveraging the complementary information between noisy...

Full description

Bibliographic Details
Main Authors: Jiaao Li, Qunbo Lv, Wenjian Zhang, Yu Zhang, Zheng Tan
Format: Article
Language:English
Published: MDPI AG 2024-03-01
Series:Sensors
Subjects:
Online Access:https://www.mdpi.com/1424-8220/24/7/2052
_version_ 1797212037391056896
author Jiaao Li
Qunbo Lv
Wenjian Zhang
Yu Zhang
Zheng Tan
author_facet Jiaao Li
Qunbo Lv
Wenjian Zhang
Yu Zhang
Zheng Tan
author_sort Jiaao Li
collection DOAJ
description Multi-frame super-resolution (MFSR) leverages complementary information between image sequences of the same scene to increase the resolution of the reconstructed image. As a branch of MFSR, burst super-resolution aims to restore image details by leveraging the complementary information between noisy sequences. In this paper, we propose an efficient burst-enhanced super-resolution network (BESR). Specifically, we introduce Geformer, a gate-enhanced transformer, and construct an enhanced CNN-Transformer block (ECTB) by combining convolutions to enhance local perception. ECTB efficiently aggregates intra-frame context and inter-frame correlation information, yielding an enhanced feature representation. Additionally, we leverage reference features to facilitate inter-frame communication, enhancing spatiotemporal coherence among multiple frames. To address the critical processes of inter-frame alignment and feature fusion, we propose optimized pyramid alignment (OPA) and hybrid feature fusion (HFF) modules to capture and utilize complementary information between multiple frames to recover more high-frequency details. Extensive experiments demonstrate that, compared to state-of-the-art methods, BESR achieves higher efficiency and competitively superior reconstruction results. On the synthetic dataset and real-world dataset of BurstSR, our BESR achieves PSNR values of 42.79 dB and 48.86 dB, respectively, outperforming other MFSR models significantly.
first_indexed 2024-04-24T10:36:01Z
format Article
id doaj.art-b9eef6f0fb17426080dee0b89975aa3d
institution Directory Open Access Journal
issn 1424-8220
language English
last_indexed 2024-04-24T10:36:01Z
publishDate 2024-03-01
publisher MDPI AG
record_format Article
series Sensors
spelling doaj.art-b9eef6f0fb17426080dee0b89975aa3d2024-04-12T13:26:05ZengMDPI AGSensors1424-82202024-03-01247205210.3390/s24072052Burst-Enhanced Super-Resolution Network (BESR)Jiaao Li0Qunbo Lv1Wenjian Zhang2Yu Zhang3Zheng Tan4Aerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, ChinaMulti-frame super-resolution (MFSR) leverages complementary information between image sequences of the same scene to increase the resolution of the reconstructed image. As a branch of MFSR, burst super-resolution aims to restore image details by leveraging the complementary information between noisy sequences. In this paper, we propose an efficient burst-enhanced super-resolution network (BESR). Specifically, we introduce Geformer, a gate-enhanced transformer, and construct an enhanced CNN-Transformer block (ECTB) by combining convolutions to enhance local perception. ECTB efficiently aggregates intra-frame context and inter-frame correlation information, yielding an enhanced feature representation. Additionally, we leverage reference features to facilitate inter-frame communication, enhancing spatiotemporal coherence among multiple frames. To address the critical processes of inter-frame alignment and feature fusion, we propose optimized pyramid alignment (OPA) and hybrid feature fusion (HFF) modules to capture and utilize complementary information between multiple frames to recover more high-frequency details. Extensive experiments demonstrate that, compared to state-of-the-art methods, BESR achieves higher efficiency and competitively superior reconstruction results. On the synthetic dataset and real-world dataset of BurstSR, our BESR achieves PSNR values of 42.79 dB and 48.86 dB, respectively, outperforming other MFSR models significantly.https://www.mdpi.com/1424-8220/24/7/2052burst super-resolutionCNN-Transformermulti-frame super-resolution
spellingShingle Jiaao Li
Qunbo Lv
Wenjian Zhang
Yu Zhang
Zheng Tan
Burst-Enhanced Super-Resolution Network (BESR)
Sensors
burst super-resolution
CNN-Transformer
multi-frame super-resolution
title Burst-Enhanced Super-Resolution Network (BESR)
title_full Burst-Enhanced Super-Resolution Network (BESR)
title_fullStr Burst-Enhanced Super-Resolution Network (BESR)
title_full_unstemmed Burst-Enhanced Super-Resolution Network (BESR)
title_short Burst-Enhanced Super-Resolution Network (BESR)
title_sort burst enhanced super resolution network besr
topic burst super-resolution
CNN-Transformer
multi-frame super-resolution
url https://www.mdpi.com/1424-8220/24/7/2052
work_keys_str_mv AT jiaaoli burstenhancedsuperresolutionnetworkbesr
AT qunbolv burstenhancedsuperresolutionnetworkbesr
AT wenjianzhang burstenhancedsuperresolutionnetworkbesr
AT yuzhang burstenhancedsuperresolutionnetworkbesr
AT zhengtan burstenhancedsuperresolutionnetworkbesr