Continuous digital zooming using generative adversarial networks for dual camera system
Abstract This paper presents a generative adversarial network (GAN) with patch match algorithm to realize a high‐quality digital zooming using two camera modules with different focal lengths. In dual camera system, shorter focal length module produces the wide‐view image with the low resolution. On...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021-10-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12274 |
| _version_ | 1797991580451209216 |
|---|---|
| author | Yifan Yang Qi Li Yongyi Yu Zhuang He Huajun Feng Zhihai Xu Yueting Chen |
| author_facet | Yifan Yang Qi Li Yongyi Yu Zhuang He Huajun Feng Zhihai Xu Yueting Chen |
| author_sort | Yifan Yang |
| collection | DOAJ |
| description | Abstract This paper presents a generative adversarial network (GAN) with patch match algorithm to realize a high‐quality digital zooming using two camera modules with different focal lengths. In dual camera system, shorter focal length module produces the wide‐view image with the low resolution. On the other hand, the longer focal length module produces the tele‐view image via optical zooming. The long‐focal image contains more details than short‐focal image and can be used to guide short‐focal image to reconstruct high frequency part. Firstly, a feature extraction block (FEB) is advanced to extract feature of long‐focal image and short focal‐image to reconstruct a wide‐view image with different resolutions. Next, a patch match algorithm is integrated into convolution neural networks (CNN) to fuse information of long‐focal with short‐focal image and generate a new fused image. Finally, the fused image and short‐focal image are merged with a feature fusion block (FFB) to predict high‐resolution images. In addition, generative adversarial networks are used for filtering information integrated by previous network and output the zoomed image. Extensive experiments on benchmark datasets show that our algorithm achieves favorable performance against state‐of‐the‐art methods. |
| first_indexed | 2024-04-11T08:53:22Z |
| format | Article |
| id | doaj.art-deb3ea6e4bd84aed9b8c86fbb641a801 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-04-11T08:53:22Z |
| publishDate | 2021-10-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-deb3ea6e4bd84aed9b8c86fbb641a8012022-12-22T04:33:21ZengWileyIET Image Processing1751-96591751-96672021-10-0115122880289010.1049/ipr2.12274Continuous digital zooming using generative adversarial networks for dual camera systemYifan Yang0Qi Li1Yongyi Yu2Zhuang He3Huajun Feng4Zhihai Xu5Yueting Chen6State Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou 310000 ChinaState Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou 310000 ChinaState Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou 310000 ChinaState Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou 310000 ChinaState Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou 310000 ChinaState Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou 310000 ChinaState Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou 310000 ChinaAbstract This paper presents a generative adversarial network (GAN) with patch match algorithm to realize a high‐quality digital zooming using two camera modules with different focal lengths. In dual camera system, shorter focal length module produces the wide‐view image with the low resolution. On the other hand, the longer focal length module produces the tele‐view image via optical zooming. The long‐focal image contains more details than short‐focal image and can be used to guide short‐focal image to reconstruct high frequency part. Firstly, a feature extraction block (FEB) is advanced to extract feature of long‐focal image and short focal‐image to reconstruct a wide‐view image with different resolutions. Next, a patch match algorithm is integrated into convolution neural networks (CNN) to fuse information of long‐focal with short‐focal image and generate a new fused image. Finally, the fused image and short‐focal image are merged with a feature fusion block (FFB) to predict high‐resolution images. In addition, generative adversarial networks are used for filtering information integrated by previous network and output the zoomed image. Extensive experiments on benchmark datasets show that our algorithm achieves favorable performance against state‐of‐the‐art methods.https://doi.org/10.1049/ipr2.12274Optical, image and video signal processingImage recognitionImage sensorsComputer vision and image processing techniques |
| spellingShingle | Yifan Yang Qi Li Yongyi Yu Zhuang He Huajun Feng Zhihai Xu Yueting Chen Continuous digital zooming using generative adversarial networks for dual camera system IET Image Processing Optical, image and video signal processing Image recognition Image sensors Computer vision and image processing techniques |
| title | Continuous digital zooming using generative adversarial networks for dual camera system |
| title_full | Continuous digital zooming using generative adversarial networks for dual camera system |
| title_fullStr | Continuous digital zooming using generative adversarial networks for dual camera system |
| title_full_unstemmed | Continuous digital zooming using generative adversarial networks for dual camera system |
| title_short | Continuous digital zooming using generative adversarial networks for dual camera system |
| title_sort | continuous digital zooming using generative adversarial networks for dual camera system |
| topic | Optical, image and video signal processing Image recognition Image sensors Computer vision and image processing techniques |
| url | https://doi.org/10.1049/ipr2.12274 |
| work_keys_str_mv | AT yifanyang continuousdigitalzoomingusinggenerativeadversarialnetworksfordualcamerasystem AT qili continuousdigitalzoomingusinggenerativeadversarialnetworksfordualcamerasystem AT yongyiyu continuousdigitalzoomingusinggenerativeadversarialnetworksfordualcamerasystem AT zhuanghe continuousdigitalzoomingusinggenerativeadversarialnetworksfordualcamerasystem AT huajunfeng continuousdigitalzoomingusinggenerativeadversarialnetworksfordualcamerasystem AT zhihaixu continuousdigitalzoomingusinggenerativeadversarialnetworksfordualcamerasystem AT yuetingchen continuousdigitalzoomingusinggenerativeadversarialnetworksfordualcamerasystem |