GANMasker: A Two-Stage Generative Adversarial Network for High-Quality Face Mask Removal
Deep-learning-based image inpainting methods have made remarkable advancements, particularly in object removal tasks. The removal of face masks has gained significant attention, especially in the wake of the COVID-19 pandemic, and while numerous methods have successfully addressed the removal of sma...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/16/7094 |
| _version_ | 1797583306681745408 |
|---|---|
| author | Mohamed Mahmoud Hyun-Soo Kang |
| author_facet | Mohamed Mahmoud Hyun-Soo Kang |
| author_sort | Mohamed Mahmoud |
| collection | DOAJ |
| description | Deep-learning-based image inpainting methods have made remarkable advancements, particularly in object removal tasks. The removal of face masks has gained significant attention, especially in the wake of the COVID-19 pandemic, and while numerous methods have successfully addressed the removal of small objects, removing large and complex masks from faces remains demanding. This paper presents a novel two-stage network for unmasking faces considering the intricate facial features typically concealed by masks, such as noses, mouths, and chins. Additionally, the scarcity of paired datasets comprising masked and unmasked face images poses an additional challenge. In the first stage of our proposed model, we employ an autoencoder-based network for binary segmentation of the face mask. Subsequently, in the second stage, we introduce a generative adversarial network (GAN)-based network enhanced with attention and Masked–Unmasked Region Fusion (MURF) mechanisms to focus on the masked region. Our network generates realistic and accurate unmasked faces that resemble the original faces. We train our model on paired unmasked and masked face images sourced from CelebA, a large public dataset, and evaluate its performance on multi-scale masked faces. The experimental results illustrate that the proposed method surpasses the current state-of-the-art techniques in both qualitative and quantitative metrics. It achieves a Peak Signal-to-Noise Ratio (PSNR) improvement of 4.18 dB over the second-best method, with the PSNR reaching 30.96. Additionally, it exhibits a 1% increase in the Structural Similarity Index Measure (SSIM), achieving a value of 0.95. |
| first_indexed | 2024-03-10T23:36:03Z |
| format | Article |
| id | doaj.art-dc3bca589d5949ce889e4ce67c195bfe |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T23:36:03Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-dc3bca589d5949ce889e4ce67c195bfe2023-11-19T02:56:44ZengMDPI AGSensors1424-82202023-08-012316709410.3390/s23167094GANMasker: A Two-Stage Generative Adversarial Network for High-Quality Face Mask RemovalMohamed Mahmoud0Hyun-Soo Kang1Department of Information and Communication Engineering, School of Electrical and Computer Engineering, Chungbuk National University, Cheongju-si 28644, Republic of KoreaDepartment of Information and Communication Engineering, School of Electrical and Computer Engineering, Chungbuk National University, Cheongju-si 28644, Republic of KoreaDeep-learning-based image inpainting methods have made remarkable advancements, particularly in object removal tasks. The removal of face masks has gained significant attention, especially in the wake of the COVID-19 pandemic, and while numerous methods have successfully addressed the removal of small objects, removing large and complex masks from faces remains demanding. This paper presents a novel two-stage network for unmasking faces considering the intricate facial features typically concealed by masks, such as noses, mouths, and chins. Additionally, the scarcity of paired datasets comprising masked and unmasked face images poses an additional challenge. In the first stage of our proposed model, we employ an autoencoder-based network for binary segmentation of the face mask. Subsequently, in the second stage, we introduce a generative adversarial network (GAN)-based network enhanced with attention and Masked–Unmasked Region Fusion (MURF) mechanisms to focus on the masked region. Our network generates realistic and accurate unmasked faces that resemble the original faces. We train our model on paired unmasked and masked face images sourced from CelebA, a large public dataset, and evaluate its performance on multi-scale masked faces. The experimental results illustrate that the proposed method surpasses the current state-of-the-art techniques in both qualitative and quantitative metrics. It achieves a Peak Signal-to-Noise Ratio (PSNR) improvement of 4.18 dB over the second-best method, with the PSNR reaching 30.96. Additionally, it exhibits a 1% increase in the Structural Similarity Index Measure (SSIM), achieving a value of 0.95.https://www.mdpi.com/1424-8220/23/16/7094face mask removalimage inpaintinggenerative adversarial networks (GANs)attention mechanismface unmaskingautoencoder |
| spellingShingle | Mohamed Mahmoud Hyun-Soo Kang GANMasker: A Two-Stage Generative Adversarial Network for High-Quality Face Mask Removal Sensors face mask removal image inpainting generative adversarial networks (GANs) attention mechanism face unmasking autoencoder |
| title | GANMasker: A Two-Stage Generative Adversarial Network for High-Quality Face Mask Removal |
| title_full | GANMasker: A Two-Stage Generative Adversarial Network for High-Quality Face Mask Removal |
| title_fullStr | GANMasker: A Two-Stage Generative Adversarial Network for High-Quality Face Mask Removal |
| title_full_unstemmed | GANMasker: A Two-Stage Generative Adversarial Network for High-Quality Face Mask Removal |
| title_short | GANMasker: A Two-Stage Generative Adversarial Network for High-Quality Face Mask Removal |
| title_sort | ganmasker a two stage generative adversarial network for high quality face mask removal |
| topic | face mask removal image inpainting generative adversarial networks (GANs) attention mechanism face unmasking autoencoder |
| url | https://www.mdpi.com/1424-8220/23/16/7094 |
| work_keys_str_mv | AT mohamedmahmoud ganmaskeratwostagegenerativeadversarialnetworkforhighqualityfacemaskremoval AT hyunsookang ganmaskeratwostagegenerativeadversarialnetworkforhighqualityfacemaskremoval |