Joint Super-Resolution and Head Pose Estimation for Extreme Low-Resolution Faces
State-of-the-art deep learning-based Head Pose Estimation (HPE) techniques have reached spectacular performance on High-Resolution (HR) face images. However, they still fail to achieve expected performance on low-resolution images at large scales. This work presents an end-to-end HPE framework assis...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10034761/ |
| _version_ | 1811169681172594688 |
|---|---|
| author | Sahar Rahimi Malakshan Mohammad Saeed Ebrahimi Saadabadi Moktari Mostofa Sobhan Soleymani Nasser M. Nasrabadi |
| author_facet | Sahar Rahimi Malakshan Mohammad Saeed Ebrahimi Saadabadi Moktari Mostofa Sobhan Soleymani Nasser M. Nasrabadi |
| author_sort | Sahar Rahimi Malakshan |
| collection | DOAJ |
| description | State-of-the-art deep learning-based Head Pose Estimation (HPE) techniques have reached spectacular performance on High-Resolution (HR) face images. However, they still fail to achieve expected performance on low-resolution images at large scales. This work presents an end-to-end HPE framework assisted by a Face Super-Resolution (FSR) algorithm. The proposed FSR model is specifically guided to enhance the HPE performance rather than considering FSR as an independent task. To this end, we utilized a Multi-Stage Generative Adversarial Network (MSGAN) which benefit from a pose-aware adversarial loss and head pose estimation feedback to generate super-resolved images that are properly aligned for HPE. Also, we propose a degradation strategy rather than simple down-sampling approach to mimic the diverse properties of real-world Low-Resolution (LR) images. We evaluate the performance of our proposed method on both synthetic and real-world LR datasets and show the superiority of our approach in both visual and HPE metrics on the AFLW2000, BIWI, and WiderFace Datasets. |
| first_indexed | 2024-04-10T16:46:15Z |
| format | Article |
| id | doaj.art-24bf138539e5430994802a06556702e6 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T16:46:15Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-24bf138539e5430994802a06556702e62023-02-08T00:00:40ZengIEEEIEEE Access2169-35362023-01-0111112381125310.1109/ACCESS.2023.324160610034761Joint Super-Resolution and Head Pose Estimation for Extreme Low-Resolution FacesSahar Rahimi Malakshan0https://orcid.org/0000-0002-6211-6039Mohammad Saeed Ebrahimi Saadabadi1https://orcid.org/0000-0003-4112-626XMoktari Mostofa2https://orcid.org/0000-0001-8719-3244Sobhan Soleymani3https://orcid.org/0000-0003-3541-0918Nasser M. Nasrabadi4https://orcid.org/0000-0001-8730-627XLane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USALane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USALane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USALane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USALane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USAState-of-the-art deep learning-based Head Pose Estimation (HPE) techniques have reached spectacular performance on High-Resolution (HR) face images. However, they still fail to achieve expected performance on low-resolution images at large scales. This work presents an end-to-end HPE framework assisted by a Face Super-Resolution (FSR) algorithm. The proposed FSR model is specifically guided to enhance the HPE performance rather than considering FSR as an independent task. To this end, we utilized a Multi-Stage Generative Adversarial Network (MSGAN) which benefit from a pose-aware adversarial loss and head pose estimation feedback to generate super-resolved images that are properly aligned for HPE. Also, we propose a degradation strategy rather than simple down-sampling approach to mimic the diverse properties of real-world Low-Resolution (LR) images. We evaluate the performance of our proposed method on both synthetic and real-world LR datasets and show the superiority of our approach in both visual and HPE metrics on the AFLW2000, BIWI, and WiderFace Datasets.https://ieeexplore.ieee.org/document/10034761/Head pose estimation (HPE)face super-resolution (FSR)multi-stage generative adversarial networks (MSGAN)low-resolution (LR) face images |
| spellingShingle | Sahar Rahimi Malakshan Mohammad Saeed Ebrahimi Saadabadi Moktari Mostofa Sobhan Soleymani Nasser M. Nasrabadi Joint Super-Resolution and Head Pose Estimation for Extreme Low-Resolution Faces IEEE Access Head pose estimation (HPE) face super-resolution (FSR) multi-stage generative adversarial networks (MSGAN) low-resolution (LR) face images |
| title | Joint Super-Resolution and Head Pose Estimation for Extreme Low-Resolution Faces |
| title_full | Joint Super-Resolution and Head Pose Estimation for Extreme Low-Resolution Faces |
| title_fullStr | Joint Super-Resolution and Head Pose Estimation for Extreme Low-Resolution Faces |
| title_full_unstemmed | Joint Super-Resolution and Head Pose Estimation for Extreme Low-Resolution Faces |
| title_short | Joint Super-Resolution and Head Pose Estimation for Extreme Low-Resolution Faces |
| title_sort | joint super resolution and head pose estimation for extreme low resolution faces |
| topic | Head pose estimation (HPE) face super-resolution (FSR) multi-stage generative adversarial networks (MSGAN) low-resolution (LR) face images |
| url | https://ieeexplore.ieee.org/document/10034761/ |
| work_keys_str_mv | AT saharrahimimalakshan jointsuperresolutionandheadposeestimationforextremelowresolutionfaces AT mohammadsaeedebrahimisaadabadi jointsuperresolutionandheadposeestimationforextremelowresolutionfaces AT moktarimostofa jointsuperresolutionandheadposeestimationforextremelowresolutionfaces AT sobhansoleymani jointsuperresolutionandheadposeestimationforextremelowresolutionfaces AT nassermnasrabadi jointsuperresolutionandheadposeestimationforextremelowresolutionfaces |