Surveillance System for Real-Time High-Precision Recognition of Criminal Faces From Wild Videos
As violent criminals, such as child sex offenders, tend to have high recidivism rates in modern society, there is a need to prevent such offenders from approaching socially disadvantaged and crime-prone areas, such as schools or childcare centers. Accordingly, national governments and related instit...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10143202/ |
| _version_ | 1797805882548944896 |
|---|---|
| author | Hyun-Bin Kim Nakhoon Choi Hye-Jeong Kwon Heeyoul Kim |
| author_facet | Hyun-Bin Kim Nakhoon Choi Hye-Jeong Kwon Heeyoul Kim |
| author_sort | Hyun-Bin Kim |
| collection | DOAJ |
| description | As violent criminals, such as child sex offenders, tend to have high recidivism rates in modern society, there is a need to prevent such offenders from approaching socially disadvantaged and crime-prone areas, such as schools or childcare centers. Accordingly, national governments and related institutions have installed surveillance cameras and provided additional personnel to manage and monitor them via video surveillance equipment. However, naked-eye monitoring by guards and manual image processing cannot properly evaluate the video captured by surveillance cameras. To address the various problems of conventional systems that simply store and retrieve image data, a system is needed that can actively classify captured images in real-time, in addition to assisting surveillance personnel. Therefore, this paper proposes a video surveillance system based on a composable deep face recognition method. The proposed system detects the faces of criminals in real time from videos captured by a surveillance camera and notifies relevant institutions of the appearance of criminals. For real-time face detection, a down-sampled image forked from the original is used to localize unspecified faces. To improve accuracy and confidence in the recognition task, a scoring method based on face tracking is proposed. The final score combines the recognition confidence and the standard score to determine the embedding distance from the criminal face embedding data. The blind spots of surveillance personnel can be effectively addressed through early detection of criminals approaching crime-prone areas. The contributions of the paper are as follows. The proposed system can process images from surveillance cameras in real-time by using down-sampling. It can effectively identify the identity of criminals by using a face tracking ID unit and minimizes prediction reversal by solving the congested embedding problem in the feature space that may occur when performing identification matching on a large amount of face embedding DBs. Additionally, the reliability of the identification results is complemented by an identification score accumulation method. In this paper, we prototyped the proposed system and experimented with the recognition model, achieving an accuracy of 0.900 and an F-1 score of 0.943. We also experimentally confirmed that the models proposed in other studies have higher performance when using the tracked instance-level face identification method proposed in this paper. It is expected that the proposed system can be used to locate criminals and protect national facilities, and such responses can quickly prevent accidents/incidents. The dataset and code are available at <italic><uri>https://github.com/aengoo/focusface</uri></italic> |
| first_indexed | 2024-03-13T05:58:50Z |
| format | Article |
| id | doaj.art-4880b1dd058c486bbe77ed0e6b8c34d7 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T05:58:50Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-4880b1dd058c486bbe77ed0e6b8c34d72023-06-12T23:02:08ZengIEEEIEEE Access2169-35362023-01-0111560665608210.1109/ACCESS.2023.328245110143202Surveillance System for Real-Time High-Precision Recognition of Criminal Faces From Wild VideosHyun-Bin Kim0Nakhoon Choi1Hye-Jeong Kwon2https://orcid.org/0000-0002-6390-0765Heeyoul Kim3https://orcid.org/0000-0001-6341-580XDepartment of Computer Science, Kyonggi University, Suwon, South KoreaDepartment of Computer Science, Kyonggi University, Suwon, South KoreaDepartment of Computer Science, Kyonggi University, Suwon, South KoreaDivision of Computer Science and Engineering, Kyonggi University, Suwon, South KoreaAs violent criminals, such as child sex offenders, tend to have high recidivism rates in modern society, there is a need to prevent such offenders from approaching socially disadvantaged and crime-prone areas, such as schools or childcare centers. Accordingly, national governments and related institutions have installed surveillance cameras and provided additional personnel to manage and monitor them via video surveillance equipment. However, naked-eye monitoring by guards and manual image processing cannot properly evaluate the video captured by surveillance cameras. To address the various problems of conventional systems that simply store and retrieve image data, a system is needed that can actively classify captured images in real-time, in addition to assisting surveillance personnel. Therefore, this paper proposes a video surveillance system based on a composable deep face recognition method. The proposed system detects the faces of criminals in real time from videos captured by a surveillance camera and notifies relevant institutions of the appearance of criminals. For real-time face detection, a down-sampled image forked from the original is used to localize unspecified faces. To improve accuracy and confidence in the recognition task, a scoring method based on face tracking is proposed. The final score combines the recognition confidence and the standard score to determine the embedding distance from the criminal face embedding data. The blind spots of surveillance personnel can be effectively addressed through early detection of criminals approaching crime-prone areas. The contributions of the paper are as follows. The proposed system can process images from surveillance cameras in real-time by using down-sampling. It can effectively identify the identity of criminals by using a face tracking ID unit and minimizes prediction reversal by solving the congested embedding problem in the feature space that may occur when performing identification matching on a large amount of face embedding DBs. Additionally, the reliability of the identification results is complemented by an identification score accumulation method. In this paper, we prototyped the proposed system and experimented with the recognition model, achieving an accuracy of 0.900 and an F-1 score of 0.943. We also experimentally confirmed that the models proposed in other studies have higher performance when using the tracked instance-level face identification method proposed in this paper. It is expected that the proposed system can be used to locate criminals and protect national facilities, and such responses can quickly prevent accidents/incidents. The dataset and code are available at <italic><uri>https://github.com/aengoo/focusface</uri></italic>https://ieeexplore.ieee.org/document/10143202/Crime preventiondown-samplingface recognitionvideo |
| spellingShingle | Hyun-Bin Kim Nakhoon Choi Hye-Jeong Kwon Heeyoul Kim Surveillance System for Real-Time High-Precision Recognition of Criminal Faces From Wild Videos IEEE Access Crime prevention down-sampling face recognition video |
| title | Surveillance System for Real-Time High-Precision Recognition of Criminal Faces From Wild Videos |
| title_full | Surveillance System for Real-Time High-Precision Recognition of Criminal Faces From Wild Videos |
| title_fullStr | Surveillance System for Real-Time High-Precision Recognition of Criminal Faces From Wild Videos |
| title_full_unstemmed | Surveillance System for Real-Time High-Precision Recognition of Criminal Faces From Wild Videos |
| title_short | Surveillance System for Real-Time High-Precision Recognition of Criminal Faces From Wild Videos |
| title_sort | surveillance system for real time high precision recognition of criminal faces from wild videos |
| topic | Crime prevention down-sampling face recognition video |
| url | https://ieeexplore.ieee.org/document/10143202/ |
| work_keys_str_mv | AT hyunbinkim surveillancesystemforrealtimehighprecisionrecognitionofcriminalfacesfromwildvideos AT nakhoonchoi surveillancesystemforrealtimehighprecisionrecognitionofcriminalfacesfromwildvideos AT hyejeongkwon surveillancesystemforrealtimehighprecisionrecognitionofcriminalfacesfromwildvideos AT heeyoulkim surveillancesystemforrealtimehighprecisionrecognitionofcriminalfacesfromwildvideos |