Attention‐based hierarchical pyramid feature fusion structure for efficient face recognition
Abstract Deep convolutional neural networks (CNN) have become the main method for face recognition (FR). To deploy deep CNN models on embedded and mobile devices, several lightweight FR models have been proposed. However, multi‐scale facial features are seldom considered in these approaches. To over...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-06-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12802 |
| _version_ | 1797813812107149312 |
|---|---|
| author | Yi Dai Kai Sun Wei Huang Dawei Zhang Gaojie Dai |
| author_facet | Yi Dai Kai Sun Wei Huang Dawei Zhang Gaojie Dai |
| author_sort | Yi Dai |
| collection | DOAJ |
| description | Abstract Deep convolutional neural networks (CNN) have become the main method for face recognition (FR). To deploy deep CNN models on embedded and mobile devices, several lightweight FR models have been proposed. However, multi‐scale facial features are seldom considered in these approaches. To overcome this limitation, an attention‐based hierarchical pyramid feature fusion (AHPF) structure was proposed in this paper. Specifically, hierarchical multi‐scale features were directly extracted from the backbone based on its pyramidal hierarchy, and the bidirectional cross‐scale connection was used to better combine the high‐level global features with low‐level local features. In addition, instead of simple concatenation or summation, an attention‐based feature fusion mechanism was used to highlight the most recognizable facial patches, and to address the unequal contribution to the output during the fusing process. Based on the AHPF structure and efficient backbones, multiple sizes of lightweight FR models were presented, called HSFNet. After an extensive experimental evaluation involving 10 mainstream benchmarks, the proposed models consistently achieved state‐of‐the‐art FR performance compared to other lightweight FR models with same level of model complexity. With only 0.659M parameters and 94.94M FLOPs, our HSFNet‐05‐M exhibited a performance competitive with recent top‐ranked FR models containing up to 4M parameters and 500M FLOPs. |
| first_indexed | 2024-03-13T07:58:13Z |
| format | Article |
| id | doaj.art-88f172ba9fb3495ca3b4e18f05603486 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-03-13T07:58:13Z |
| publishDate | 2023-06-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-88f172ba9fb3495ca3b4e18f056034862023-06-02T03:06:38ZengWileyIET Image Processing1751-96591751-96672023-06-011782399240910.1049/ipr2.12802Attention‐based hierarchical pyramid feature fusion structure for efficient face recognitionYi Dai0Kai Sun1Wei Huang2Dawei Zhang3Gaojie Dai4School of Electronic Information Engineering Inner Mongolia University Hohhot People's Republic of ChinaSchool of Electronic Information Engineering Inner Mongolia University Hohhot People's Republic of ChinaSchool of Electronic Information Engineering Inner Mongolia University Hohhot People's Republic of ChinaState Grid Yichuan County Electric Power Supply Branch Yichuan People's Republic of ChinaState Grid Yichuan County Electric Power Supply Branch Yichuan People's Republic of ChinaAbstract Deep convolutional neural networks (CNN) have become the main method for face recognition (FR). To deploy deep CNN models on embedded and mobile devices, several lightweight FR models have been proposed. However, multi‐scale facial features are seldom considered in these approaches. To overcome this limitation, an attention‐based hierarchical pyramid feature fusion (AHPF) structure was proposed in this paper. Specifically, hierarchical multi‐scale features were directly extracted from the backbone based on its pyramidal hierarchy, and the bidirectional cross‐scale connection was used to better combine the high‐level global features with low‐level local features. In addition, instead of simple concatenation or summation, an attention‐based feature fusion mechanism was used to highlight the most recognizable facial patches, and to address the unequal contribution to the output during the fusing process. Based on the AHPF structure and efficient backbones, multiple sizes of lightweight FR models were presented, called HSFNet. After an extensive experimental evaluation involving 10 mainstream benchmarks, the proposed models consistently achieved state‐of‐the‐art FR performance compared to other lightweight FR models with same level of model complexity. With only 0.659M parameters and 94.94M FLOPs, our HSFNet‐05‐M exhibited a performance competitive with recent top‐ranked FR models containing up to 4M parameters and 500M FLOPs.https://doi.org/10.1049/ipr2.12802computer visionface recognitionfeature extraction |
| spellingShingle | Yi Dai Kai Sun Wei Huang Dawei Zhang Gaojie Dai Attention‐based hierarchical pyramid feature fusion structure for efficient face recognition IET Image Processing computer vision face recognition feature extraction |
| title | Attention‐based hierarchical pyramid feature fusion structure for efficient face recognition |
| title_full | Attention‐based hierarchical pyramid feature fusion structure for efficient face recognition |
| title_fullStr | Attention‐based hierarchical pyramid feature fusion structure for efficient face recognition |
| title_full_unstemmed | Attention‐based hierarchical pyramid feature fusion structure for efficient face recognition |
| title_short | Attention‐based hierarchical pyramid feature fusion structure for efficient face recognition |
| title_sort | attention based hierarchical pyramid feature fusion structure for efficient face recognition |
| topic | computer vision face recognition feature extraction |
| url | https://doi.org/10.1049/ipr2.12802 |
| work_keys_str_mv | AT yidai attentionbasedhierarchicalpyramidfeaturefusionstructureforefficientfacerecognition AT kaisun attentionbasedhierarchicalpyramidfeaturefusionstructureforefficientfacerecognition AT weihuang attentionbasedhierarchicalpyramidfeaturefusionstructureforefficientfacerecognition AT daweizhang attentionbasedhierarchicalpyramidfeaturefusionstructureforefficientfacerecognition AT gaojiedai attentionbasedhierarchicalpyramidfeaturefusionstructureforefficientfacerecognition |