Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. Whisper
The growth in online child exploitation material is a significant challenge for European Law Enforcement Agencies (LEAs). One of the most important sources of such online information corresponds to audio material that needs to be analyzed to find evidence in a timely and practical manner. That is wh...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/4/1843 |
| _version_ | 1827755630621884416 |
|---|---|
| author | Juan Camilo Vásquez-Correa Aitor Álvarez Muniain |
| author_facet | Juan Camilo Vásquez-Correa Aitor Álvarez Muniain |
| author_sort | Juan Camilo Vásquez-Correa |
| collection | DOAJ |
| description | The growth in online child exploitation material is a significant challenge for European Law Enforcement Agencies (LEAs). One of the most important sources of such online information corresponds to audio material that needs to be analyzed to find evidence in a timely and practical manner. That is why LEAs require a next-generation AI-powered platform to process audio data from online sources. We propose the use of speech recognition and keyword spotting to transcribe audiovisual data and to detect the presence of keywords related to child abuse. The considered models are based on two of the most accurate neural-based architectures to date: Wav2vec2.0 and Whisper. The systems were tested under an extensive set of scenarios in different languages. Additionally, keeping in mind that obtaining data from LEAs are very sensitive, we explore the use of federated learning to provide more robust systems for the addressed application, while maintaining the privacy of the data from LEAs. The considered models achieved a word error rate between 11% and 25%, depending on the language. In addition, the systems are able to recognize a set of spotted words with true-positive rates between 82% and 98%, depending on the language. Finally, federated learning strategies show that they can maintain and even improve the performance of the systems when compared to centralized trained models. The proposed systems set the basis for an AI-powered platform for automatic analysis of audio in the context of forensic applications of child abuse. The use of federated learning is also promising for the addressed scenario, where data privacy is an important issue to be managed. |
| first_indexed | 2024-03-11T08:11:43Z |
| format | Article |
| id | doaj.art-4af1452246ad4984880885967162c397 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T08:11:43Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-4af1452246ad4984880885967162c3972023-11-16T23:06:52ZengMDPI AGSensors1424-82202023-02-01234184310.3390/s23041843Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. WhisperJuan Camilo Vásquez-Correa0Aitor Álvarez Muniain1Fundación Vicomtech, Basque Research and Technology Alliance (BRTA), Mikeletegi 57, 20009 Donostia-San Sebastián, SpainFundación Vicomtech, Basque Research and Technology Alliance (BRTA), Mikeletegi 57, 20009 Donostia-San Sebastián, SpainThe growth in online child exploitation material is a significant challenge for European Law Enforcement Agencies (LEAs). One of the most important sources of such online information corresponds to audio material that needs to be analyzed to find evidence in a timely and practical manner. That is why LEAs require a next-generation AI-powered platform to process audio data from online sources. We propose the use of speech recognition and keyword spotting to transcribe audiovisual data and to detect the presence of keywords related to child abuse. The considered models are based on two of the most accurate neural-based architectures to date: Wav2vec2.0 and Whisper. The systems were tested under an extensive set of scenarios in different languages. Additionally, keeping in mind that obtaining data from LEAs are very sensitive, we explore the use of federated learning to provide more robust systems for the addressed application, while maintaining the privacy of the data from LEAs. The considered models achieved a word error rate between 11% and 25%, depending on the language. In addition, the systems are able to recognize a set of spotted words with true-positive rates between 82% and 98%, depending on the language. Finally, federated learning strategies show that they can maintain and even improve the performance of the systems when compared to centralized trained models. The proposed systems set the basis for an AI-powered platform for automatic analysis of audio in the context of forensic applications of child abuse. The use of federated learning is also promising for the addressed scenario, where data privacy is an important issue to be managed.https://www.mdpi.com/1424-8220/23/4/1843speech recognitionkeyword spottingchild abusefederated learningWhisperWav2vec2.0 |
| spellingShingle | Juan Camilo Vásquez-Correa Aitor Álvarez Muniain Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. Whisper Sensors speech recognition keyword spotting child abuse federated learning Whisper Wav2vec2.0 |
| title | Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. Whisper |
| title_full | Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. Whisper |
| title_fullStr | Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. Whisper |
| title_full_unstemmed | Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. Whisper |
| title_short | Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. Whisper |
| title_sort | novel speech recognition systems applied to forensics within child exploitation wav2vec2 0 vs whisper |
| topic | speech recognition keyword spotting child abuse federated learning Whisper Wav2vec2.0 |
| url | https://www.mdpi.com/1424-8220/23/4/1843 |
| work_keys_str_mv | AT juancamilovasquezcorrea novelspeechrecognitionsystemsappliedtoforensicswithinchildexploitationwav2vec20vswhisper AT aitoralvarezmuniain novelspeechrecognitionsystemsappliedtoforensicswithinchildexploitationwav2vec20vswhisper |