Facial image super-resolution guided by adaptive geometric features
Abstract This paper addresses the traditional issue of restoring a high-resolution (HR) facial image from a low-resolution (LR) counterpart. Current state-of-the-art super-resolution (SR) methods commonly adopt the convolutional neural networks to learn a non-linear complex mapping between paired LR...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2020-07-01
|
| Series: | EURASIP Journal on Wireless Communications and Networking |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s13638-020-01760-y |
| _version_ | 1818180601683902464 |
|---|---|
| author | Zhenfeng Fan Xiyuan Hu Chen Chen Xiaolian Wang Silong Peng |
| author_facet | Zhenfeng Fan Xiyuan Hu Chen Chen Xiaolian Wang Silong Peng |
| author_sort | Zhenfeng Fan |
| collection | DOAJ |
| description | Abstract This paper addresses the traditional issue of restoring a high-resolution (HR) facial image from a low-resolution (LR) counterpart. Current state-of-the-art super-resolution (SR) methods commonly adopt the convolutional neural networks to learn a non-linear complex mapping between paired LR and HR images. They discriminate local patterns expressed by the neighboring pixels along the planar directions but ignore the intrinsic 3D proximity including the depth map. As a special case of general images, the face has limited geometric variations, which we believe that the relevant depth map can be learned and used to guide the face SR task. Motivated by it, we design a network including two branches: one for auxiliary depth map estimation and the other for the main SR task. Adaptive geometric features are further learned from the depth map and used to modulate the mid-level features of the SR branch. The whole network is implemented in an end-to-end trainable manner under the extra supervision of depth map. The supervisory depth map is either a paired one from RGB-D scans or a reconstructed one by a 3D prior model of faces. The experiments demonstrate the effectiveness of the proposed method and achieve improved performance over the state of the arts. |
| first_indexed | 2024-12-11T21:22:22Z |
| format | Article |
| id | doaj.art-165648dad7d54de687fc3bfd153d4203 |
| institution | Directory Open Access Journal |
| issn | 1687-1499 |
| language | English |
| last_indexed | 2024-12-11T21:22:22Z |
| publishDate | 2020-07-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | EURASIP Journal on Wireless Communications and Networking |
| spelling | doaj.art-165648dad7d54de687fc3bfd153d42032022-12-22T00:50:25ZengSpringerOpenEURASIP Journal on Wireless Communications and Networking1687-14992020-07-012020111510.1186/s13638-020-01760-yFacial image super-resolution guided by adaptive geometric featuresZhenfeng Fan0Xiyuan Hu1Chen Chen2Xiaolian Wang3Silong Peng4Institute of Automation, Chinese Academy of SciencesInstitute of Automation, Chinese Academy of SciencesInstitute of Automation, Chinese Academy of SciencesInstitute of Automation, Chinese Academy of SciencesInstitute of Automation, Chinese Academy of SciencesAbstract This paper addresses the traditional issue of restoring a high-resolution (HR) facial image from a low-resolution (LR) counterpart. Current state-of-the-art super-resolution (SR) methods commonly adopt the convolutional neural networks to learn a non-linear complex mapping between paired LR and HR images. They discriminate local patterns expressed by the neighboring pixels along the planar directions but ignore the intrinsic 3D proximity including the depth map. As a special case of general images, the face has limited geometric variations, which we believe that the relevant depth map can be learned and used to guide the face SR task. Motivated by it, we design a network including two branches: one for auxiliary depth map estimation and the other for the main SR task. Adaptive geometric features are further learned from the depth map and used to modulate the mid-level features of the SR branch. The whole network is implemented in an end-to-end trainable manner under the extra supervision of depth map. The supervisory depth map is either a paired one from RGB-D scans or a reconstructed one by a 3D prior model of faces. The experiments demonstrate the effectiveness of the proposed method and achieve improved performance over the state of the arts.http://link.springer.com/article/10.1186/s13638-020-01760-yConvolutional neural networks (CNNs)Depth mapFace super-resolution |
| spellingShingle | Zhenfeng Fan Xiyuan Hu Chen Chen Xiaolian Wang Silong Peng Facial image super-resolution guided by adaptive geometric features EURASIP Journal on Wireless Communications and Networking Convolutional neural networks (CNNs) Depth map Face super-resolution |
| title | Facial image super-resolution guided by adaptive geometric features |
| title_full | Facial image super-resolution guided by adaptive geometric features |
| title_fullStr | Facial image super-resolution guided by adaptive geometric features |
| title_full_unstemmed | Facial image super-resolution guided by adaptive geometric features |
| title_short | Facial image super-resolution guided by adaptive geometric features |
| title_sort | facial image super resolution guided by adaptive geometric features |
| topic | Convolutional neural networks (CNNs) Depth map Face super-resolution |
| url | http://link.springer.com/article/10.1186/s13638-020-01760-y |
| work_keys_str_mv | AT zhenfengfan facialimagesuperresolutionguidedbyadaptivegeometricfeatures AT xiyuanhu facialimagesuperresolutionguidedbyadaptivegeometricfeatures AT chenchen facialimagesuperresolutionguidedbyadaptivegeometricfeatures AT xiaolianwang facialimagesuperresolutionguidedbyadaptivegeometricfeatures AT silongpeng facialimagesuperresolutionguidedbyadaptivegeometricfeatures |