Multilingual Audio-Visual Smartphone Dataset and Evaluation
Smartphones have been employed with biometric-based verification systems to provide security in highly sensitive applications. Audio-visual biometrics are getting popular due to their usability, and also it will be challenging to spoof because of their multimodal nature. In this work, we present an...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9600884/ |
| _version_ | 1819000893509468160 |
|---|---|
| author | Hareesh Mandalapu P. N. Aravinda Reddy Raghavendra Ramachandra Krothapalli Sreenivasa Rao Pabitra Mitra S. R. Mahadeva Prasanna Christoph Busch |
| author_facet | Hareesh Mandalapu P. N. Aravinda Reddy Raghavendra Ramachandra Krothapalli Sreenivasa Rao Pabitra Mitra S. R. Mahadeva Prasanna Christoph Busch |
| author_sort | Hareesh Mandalapu |
| collection | DOAJ |
| description | Smartphones have been employed with biometric-based verification systems to provide security in highly sensitive applications. Audio-visual biometrics are getting popular due to their usability, and also it will be challenging to spoof because of their multimodal nature. In this work, we present an audio-visual smartphone dataset captured in five different recent smartphones. This new dataset contains 103 subjects captured in three different sessions considering the different real-world scenarios. Three different languages are acquired in this dataset to include the problem of language dependency of the speaker recognition systems. These unique characteristics of this dataset will pave the way to implement novel state-of-the-art unimodal or audio-visual speaker recognition systems. We also report the performance of the bench-marked biometric verification systems on our dataset. The robustness of biometric algorithms is evaluated towards multiple dependencies like signal noise, device, language and presentation attacks like replay and synthesized signals with extensive experiments. The obtained results raised many concerns about the generalization properties of state-of-the-art biometrics methods in smartphones. |
| first_indexed | 2024-12-20T22:40:33Z |
| format | Article |
| id | doaj.art-884f705a0b474dd0b570d8f728e89711 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-20T22:40:33Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-884f705a0b474dd0b570d8f728e897112022-12-21T19:24:29ZengIEEEIEEE Access2169-35362021-01-01915324015325710.1109/ACCESS.2021.31254859600884Multilingual Audio-Visual Smartphone Dataset and EvaluationHareesh Mandalapu0https://orcid.org/0000-0002-2760-2832P. N. Aravinda Reddy1Raghavendra Ramachandra2https://orcid.org/0000-0003-0484-3956Krothapalli Sreenivasa Rao3Pabitra Mitra4S. R. Mahadeva Prasanna5Christoph Busch6https://orcid.org/0000-0002-9159-2923Department of Information Security and Communication Technology, Norwegian University of Science and Technology (NTNU), Gjøvik, NorwayAdvanced Technology Development Centre, IIT Kharagpur, Kharagpur, West Bengal, IndiaDepartment of Information Security and Communication Technology, Norwegian University of Science and Technology (NTNU), Gjøvik, NorwayDepartment of Computer Science and Engineering, IIT Kharagpur, Kharagpur, West Bengal, IndiaDepartment of Computer Science and Engineering, IIT Kharagpur, Kharagpur, West Bengal, IndiaDepartment of Electrical Engineering, IIT Dharwad, Dharwad, Karnataka, IndiaDepartment of Information Security and Communication Technology, Norwegian University of Science and Technology (NTNU), Gjøvik, NorwaySmartphones have been employed with biometric-based verification systems to provide security in highly sensitive applications. Audio-visual biometrics are getting popular due to their usability, and also it will be challenging to spoof because of their multimodal nature. In this work, we present an audio-visual smartphone dataset captured in five different recent smartphones. This new dataset contains 103 subjects captured in three different sessions considering the different real-world scenarios. Three different languages are acquired in this dataset to include the problem of language dependency of the speaker recognition systems. These unique characteristics of this dataset will pave the way to implement novel state-of-the-art unimodal or audio-visual speaker recognition systems. We also report the performance of the bench-marked biometric verification systems on our dataset. The robustness of biometric algorithms is evaluated towards multiple dependencies like signal noise, device, language and presentation attacks like replay and synthesized signals with extensive experiments. The obtained results raised many concerns about the generalization properties of state-of-the-art biometrics methods in smartphones.https://ieeexplore.ieee.org/document/9600884/Smartphone biometricsaudio-visual speaker recognitionpresentation attack detectionmultilingual |
| spellingShingle | Hareesh Mandalapu P. N. Aravinda Reddy Raghavendra Ramachandra Krothapalli Sreenivasa Rao Pabitra Mitra S. R. Mahadeva Prasanna Christoph Busch Multilingual Audio-Visual Smartphone Dataset and Evaluation IEEE Access Smartphone biometrics audio-visual speaker recognition presentation attack detection multilingual |
| title | Multilingual Audio-Visual Smartphone Dataset and Evaluation |
| title_full | Multilingual Audio-Visual Smartphone Dataset and Evaluation |
| title_fullStr | Multilingual Audio-Visual Smartphone Dataset and Evaluation |
| title_full_unstemmed | Multilingual Audio-Visual Smartphone Dataset and Evaluation |
| title_short | Multilingual Audio-Visual Smartphone Dataset and Evaluation |
| title_sort | multilingual audio visual smartphone dataset and evaluation |
| topic | Smartphone biometrics audio-visual speaker recognition presentation attack detection multilingual |
| url | https://ieeexplore.ieee.org/document/9600884/ |
| work_keys_str_mv | AT hareeshmandalapu multilingualaudiovisualsmartphonedatasetandevaluation AT pnaravindareddy multilingualaudiovisualsmartphonedatasetandevaluation AT raghavendraramachandra multilingualaudiovisualsmartphonedatasetandevaluation AT krothapallisreenivasarao multilingualaudiovisualsmartphonedatasetandevaluation AT pabitramitra multilingualaudiovisualsmartphonedatasetandevaluation AT srmahadevaprasanna multilingualaudiovisualsmartphonedatasetandevaluation AT christophbusch multilingualaudiovisualsmartphonedatasetandevaluation |