Multimodal Biometrics Recognition Using a Deep Convolutional Neural Network with Transfer Learning in Surveillance Videos
Biometric recognition is a critical task in security control systems. Although the face has long been widely accepted as a practical biometric for human recognition, it can be easily stolen and imitated. Moreover, in video surveillance, it is a challenge to obtain reliable facial information from an...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-07-01
|
| Series: | Computation |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-3197/10/7/127 |
| _version_ | 1797440151794745344 |
|---|---|
| author | Hsu Mon Lei Aung Charnchai Pluempitiwiriyawej Kazuhiko Hamamoto Somkiat Wangsiripitak |
| author_facet | Hsu Mon Lei Aung Charnchai Pluempitiwiriyawej Kazuhiko Hamamoto Somkiat Wangsiripitak |
| author_sort | Hsu Mon Lei Aung |
| collection | DOAJ |
| description | Biometric recognition is a critical task in security control systems. Although the face has long been widely accepted as a practical biometric for human recognition, it can be easily stolen and imitated. Moreover, in video surveillance, it is a challenge to obtain reliable facial information from an image taken at a long distance with a low-resolution camera. Gait, on the other hand, has been recently used for human recognition because gait is not easy to replicate, and reliable information can be obtained from a low-resolution camera at a long distance. However, the gait biometric alone still has constraints due to its intrinsic factors. In this paper, we propose a multimodal biometrics system by combining information from both the face and gait. Our proposed system uses a deep convolutional neural network with transfer learning. Our proposed network model learns discriminative spatiotemporal features from gait and facial features from face images. The two extracted features are fused into a common feature space at the feature level. This study conducted experiments on the publicly available CASIA-B gait and Extended Yale-B databases and a dataset of walking videos of 25 users. The proposed model achieves a 97.3 percent classification accuracy with an F1 score of 0.97and an equal error rate (EER) of 0.004. |
| first_indexed | 2024-03-09T12:03:59Z |
| format | Article |
| id | doaj.art-b360fde833164251b30c28bc485f6f7d |
| institution | Directory Open Access Journal |
| issn | 2079-3197 |
| language | English |
| last_indexed | 2024-03-09T12:03:59Z |
| publishDate | 2022-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Computation |
| spelling | doaj.art-b360fde833164251b30c28bc485f6f7d2023-11-30T23:00:42ZengMDPI AGComputation2079-31972022-07-0110712710.3390/computation10070127Multimodal Biometrics Recognition Using a Deep Convolutional Neural Network with Transfer Learning in Surveillance VideosHsu Mon Lei Aung0Charnchai Pluempitiwiriyawej1Kazuhiko Hamamoto2Somkiat Wangsiripitak3Department of Electrical Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok 10330, ThailandMultimedia Data Analytics and Processing Research Unit, Department of Electrical Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok 10330, ThailandSchool of Information and Telecommunication Engineering, Tokai University, Tokyo 108-8619, JapanFaculty of Information Technology, King Mongkut’s Institute of Technology Ladkrabang, Bangkok 10520, ThailandBiometric recognition is a critical task in security control systems. Although the face has long been widely accepted as a practical biometric for human recognition, it can be easily stolen and imitated. Moreover, in video surveillance, it is a challenge to obtain reliable facial information from an image taken at a long distance with a low-resolution camera. Gait, on the other hand, has been recently used for human recognition because gait is not easy to replicate, and reliable information can be obtained from a low-resolution camera at a long distance. However, the gait biometric alone still has constraints due to its intrinsic factors. In this paper, we propose a multimodal biometrics system by combining information from both the face and gait. Our proposed system uses a deep convolutional neural network with transfer learning. Our proposed network model learns discriminative spatiotemporal features from gait and facial features from face images. The two extracted features are fused into a common feature space at the feature level. This study conducted experiments on the publicly available CASIA-B gait and Extended Yale-B databases and a dataset of walking videos of 25 users. The proposed model achieves a 97.3 percent classification accuracy with an F1 score of 0.97and an equal error rate (EER) of 0.004.https://www.mdpi.com/2079-3197/10/7/127multimodal biometricshuman recognitiondeep CNNstransfer learning |
| spellingShingle | Hsu Mon Lei Aung Charnchai Pluempitiwiriyawej Kazuhiko Hamamoto Somkiat Wangsiripitak Multimodal Biometrics Recognition Using a Deep Convolutional Neural Network with Transfer Learning in Surveillance Videos Computation multimodal biometrics human recognition deep CNNs transfer learning |
| title | Multimodal Biometrics Recognition Using a Deep Convolutional Neural Network with Transfer Learning in Surveillance Videos |
| title_full | Multimodal Biometrics Recognition Using a Deep Convolutional Neural Network with Transfer Learning in Surveillance Videos |
| title_fullStr | Multimodal Biometrics Recognition Using a Deep Convolutional Neural Network with Transfer Learning in Surveillance Videos |
| title_full_unstemmed | Multimodal Biometrics Recognition Using a Deep Convolutional Neural Network with Transfer Learning in Surveillance Videos |
| title_short | Multimodal Biometrics Recognition Using a Deep Convolutional Neural Network with Transfer Learning in Surveillance Videos |
| title_sort | multimodal biometrics recognition using a deep convolutional neural network with transfer learning in surveillance videos |
| topic | multimodal biometrics human recognition deep CNNs transfer learning |
| url | https://www.mdpi.com/2079-3197/10/7/127 |
| work_keys_str_mv | AT hsumonleiaung multimodalbiometricsrecognitionusingadeepconvolutionalneuralnetworkwithtransferlearninginsurveillancevideos AT charnchaipluempitiwiriyawej multimodalbiometricsrecognitionusingadeepconvolutionalneuralnetworkwithtransferlearninginsurveillancevideos AT kazuhikohamamoto multimodalbiometricsrecognitionusingadeepconvolutionalneuralnetworkwithtransferlearninginsurveillancevideos AT somkiatwangsiripitak multimodalbiometricsrecognitionusingadeepconvolutionalneuralnetworkwithtransferlearninginsurveillancevideos |