AFD-StackGAN: Automatic Mask Generation Network for Face De-Occlusion Using StackGAN
To address the problem of automatically detecting and removing the mask without user interaction, we present a GAN-based automatic approach for face de-occlusion, called Automatic Mask Generation Network for Face De-occlusion Using Stacked Generative Adversarial Networks (AFD-StackGAN). In this appr...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/5/1747 |
| _version_ | 1797473771657887744 |
|---|---|
| author | Abdul Jabbar Xi Li Muhammad Assam Javed Ali Khan Marwa Obayya Mimouna Abdullah Alkhonaini Fahd N. Al-Wesabi Muhammad Assad |
| author_facet | Abdul Jabbar Xi Li Muhammad Assam Javed Ali Khan Marwa Obayya Mimouna Abdullah Alkhonaini Fahd N. Al-Wesabi Muhammad Assad |
| author_sort | Abdul Jabbar |
| collection | DOAJ |
| description | To address the problem of automatically detecting and removing the mask without user interaction, we present a GAN-based automatic approach for face de-occlusion, called Automatic Mask Generation Network for Face De-occlusion Using Stacked Generative Adversarial Networks (AFD-StackGAN). In this approach, we decompose the problem into two primary stages (i.e., Stage-I Network and Stage-II Network) and employ a separate GAN in both stages. Stage-I Network (Binary Mask Generation Network) automatically creates a binary mask for the masked region in the input images (occluded images). Then, Stage-II Network (Face De-occlusion Network) removes the mask object and synthesizes the damaged region with fine details while retaining the restored face’s appearance and structural consistency. Furthermore, we create a paired synthetic face-occluded dataset using the publicly available CelebA face images to train the proposed model. AFD-StackGAN is evaluated using real-world test images gathered from the Internet. Our extensive experimental results confirm the robustness and efficiency of the proposed model in removing complex mask objects from facial images compared to the previous image manipulation approaches. Additionally, we provide ablation studies for performance comparison between the user-defined mask and auto-defined mask and demonstrate the benefits of refiner networks in the generation process. |
| first_indexed | 2024-03-09T20:22:10Z |
| format | Article |
| id | doaj.art-356dc88a9cba48f39c4ddd525d7b9698 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T20:22:10Z |
| publishDate | 2022-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-356dc88a9cba48f39c4ddd525d7b96982023-11-23T23:45:35ZengMDPI AGSensors1424-82202022-02-01225174710.3390/s22051747AFD-StackGAN: Automatic Mask Generation Network for Face De-Occlusion Using StackGANAbdul Jabbar0Xi Li1Muhammad Assam2Javed Ali Khan3Marwa Obayya4Mimouna Abdullah Alkhonaini5Fahd N. Al-Wesabi6Muhammad Assad7College of Computer Science, Zhejiang University, Hangzhou 310027, ChinaCollege of Computer Science, Zhejiang University, Hangzhou 310027, ChinaCollege of Computer Science, Zhejiang University, Hangzhou 310027, ChinaDepartment of Software Engineering, University of Science and Technology, Bunnu 28100, PakistanDepartment of Biomedical Engineering, College of Engineering, Princess Nourah Bint Abdulrahman University, Riyadh 11671, Saudi ArabiaDepartment of Computer Science, College of Computer and Information Sciences, Prince Sultan University, Riyadh 12435, Saudi ArabiaDepartment of Computer Science, College of Science & Art at Mahayil, King Khalid University, Abha 62529, Saudi ArabiaInstitute for Frontier Materials, Deakin University, Geelong, VIC 3216, AustraliaTo address the problem of automatically detecting and removing the mask without user interaction, we present a GAN-based automatic approach for face de-occlusion, called Automatic Mask Generation Network for Face De-occlusion Using Stacked Generative Adversarial Networks (AFD-StackGAN). In this approach, we decompose the problem into two primary stages (i.e., Stage-I Network and Stage-II Network) and employ a separate GAN in both stages. Stage-I Network (Binary Mask Generation Network) automatically creates a binary mask for the masked region in the input images (occluded images). Then, Stage-II Network (Face De-occlusion Network) removes the mask object and synthesizes the damaged region with fine details while retaining the restored face’s appearance and structural consistency. Furthermore, we create a paired synthetic face-occluded dataset using the publicly available CelebA face images to train the proposed model. AFD-StackGAN is evaluated using real-world test images gathered from the Internet. Our extensive experimental results confirm the robustness and efficiency of the proposed model in removing complex mask objects from facial images compared to the previous image manipulation approaches. Additionally, we provide ablation studies for performance comparison between the user-defined mask and auto-defined mask and demonstrate the benefits of refiner networks in the generation process.https://www.mdpi.com/1424-8220/22/5/1747generative adversarial network (GAN)automatic mask removalimage restoration |
| spellingShingle | Abdul Jabbar Xi Li Muhammad Assam Javed Ali Khan Marwa Obayya Mimouna Abdullah Alkhonaini Fahd N. Al-Wesabi Muhammad Assad AFD-StackGAN: Automatic Mask Generation Network for Face De-Occlusion Using StackGAN Sensors generative adversarial network (GAN) automatic mask removal image restoration |
| title | AFD-StackGAN: Automatic Mask Generation Network for Face De-Occlusion Using StackGAN |
| title_full | AFD-StackGAN: Automatic Mask Generation Network for Face De-Occlusion Using StackGAN |
| title_fullStr | AFD-StackGAN: Automatic Mask Generation Network for Face De-Occlusion Using StackGAN |
| title_full_unstemmed | AFD-StackGAN: Automatic Mask Generation Network for Face De-Occlusion Using StackGAN |
| title_short | AFD-StackGAN: Automatic Mask Generation Network for Face De-Occlusion Using StackGAN |
| title_sort | afd stackgan automatic mask generation network for face de occlusion using stackgan |
| topic | generative adversarial network (GAN) automatic mask removal image restoration |
| url | https://www.mdpi.com/1424-8220/22/5/1747 |
| work_keys_str_mv | AT abduljabbar afdstackganautomaticmaskgenerationnetworkforfacedeocclusionusingstackgan AT xili afdstackganautomaticmaskgenerationnetworkforfacedeocclusionusingstackgan AT muhammadassam afdstackganautomaticmaskgenerationnetworkforfacedeocclusionusingstackgan AT javedalikhan afdstackganautomaticmaskgenerationnetworkforfacedeocclusionusingstackgan AT marwaobayya afdstackganautomaticmaskgenerationnetworkforfacedeocclusionusingstackgan AT mimounaabdullahalkhonaini afdstackganautomaticmaskgenerationnetworkforfacedeocclusionusingstackgan AT fahdnalwesabi afdstackganautomaticmaskgenerationnetworkforfacedeocclusionusingstackgan AT muhammadassad afdstackganautomaticmaskgenerationnetworkforfacedeocclusionusingstackgan |