Audio deepfakes: A survey
A deepfake is content or material that is synthetically generated or manipulated using artificial intelligence (AI) methods, to be passed off as real and can include audio, video, image, and text synthesis. The key difference between manual editing and deepfakes is that deepfakes are AI generated or...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-01-01
|
| Series: | Frontiers in Big Data |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fdata.2022.1001063/full |
| _version_ | 1797956657513234432 |
|---|---|
| author | Zahra Khanjani Gabrielle Watson Vandana P. Janeja |
| author_facet | Zahra Khanjani Gabrielle Watson Vandana P. Janeja |
| author_sort | Zahra Khanjani |
| collection | DOAJ |
| description | A deepfake is content or material that is synthetically generated or manipulated using artificial intelligence (AI) methods, to be passed off as real and can include audio, video, image, and text synthesis. The key difference between manual editing and deepfakes is that deepfakes are AI generated or AI manipulated and closely resemble authentic artifacts. In some cases, deepfakes can be fabricated using AI-generated content in its entirety. Deepfakes have started to have a major impact on society with more generation mechanisms emerging everyday. This article makes a contribution in understanding the landscape of deepfakes, and their detection and generation methods. We evaluate various categories of deepfakes especially in audio. The purpose of this survey is to provide readers with a deeper understanding of (1) different deepfake categories; (2) how they could be created and detected; (3) more specifically, how audio deepfakes are created and detected in more detail, which is the main focus of this paper. We found that generative adversarial networks (GANs), convolutional neural networks (CNNs), and deep neural networks (DNNs) are common ways of creating and detecting deepfakes. In our evaluation of over 150 methods, we found that the majority of the focus is on video deepfakes, and, in particular, the generation of video deepfakes. We found that for text deepfakes, there are more generation methods but very few robust methods for detection, including fake news detection, which has become a controversial area of research because of the potential heavy overlaps with human generation of fake content. Our study reveals a clear need to research audio deepfakes and particularly detection of audio deepfakes. This survey has been conducted with a different perspective, compared to existing survey papers that mostly focus on just video and image deepfakes. This survey mainly focuses on audio deepfakes that are overlooked in most of the existing surveys. This article's most important contribution is to critically analyze and provide a unique source of audio deepfake research, mostly ranging from 2016 to 2021. To the best of our knowledge, this is the first survey focusing on audio deepfakes generation and detection in English. |
| first_indexed | 2024-04-10T23:52:17Z |
| format | Article |
| id | doaj.art-44f4cfd0d68b448780d3c3c0532fd9f8 |
| institution | Directory Open Access Journal |
| issn | 2624-909X |
| language | English |
| last_indexed | 2024-04-10T23:52:17Z |
| publishDate | 2023-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Big Data |
| spelling | doaj.art-44f4cfd0d68b448780d3c3c0532fd9f82023-01-10T15:13:21ZengFrontiers Media S.A.Frontiers in Big Data2624-909X2023-01-01510.3389/fdata.2022.10010631001063Audio deepfakes: A surveyZahra KhanjaniGabrielle WatsonVandana P. JanejaA deepfake is content or material that is synthetically generated or manipulated using artificial intelligence (AI) methods, to be passed off as real and can include audio, video, image, and text synthesis. The key difference between manual editing and deepfakes is that deepfakes are AI generated or AI manipulated and closely resemble authentic artifacts. In some cases, deepfakes can be fabricated using AI-generated content in its entirety. Deepfakes have started to have a major impact on society with more generation mechanisms emerging everyday. This article makes a contribution in understanding the landscape of deepfakes, and their detection and generation methods. We evaluate various categories of deepfakes especially in audio. The purpose of this survey is to provide readers with a deeper understanding of (1) different deepfake categories; (2) how they could be created and detected; (3) more specifically, how audio deepfakes are created and detected in more detail, which is the main focus of this paper. We found that generative adversarial networks (GANs), convolutional neural networks (CNNs), and deep neural networks (DNNs) are common ways of creating and detecting deepfakes. In our evaluation of over 150 methods, we found that the majority of the focus is on video deepfakes, and, in particular, the generation of video deepfakes. We found that for text deepfakes, there are more generation methods but very few robust methods for detection, including fake news detection, which has become a controversial area of research because of the potential heavy overlaps with human generation of fake content. Our study reveals a clear need to research audio deepfakes and particularly detection of audio deepfakes. This survey has been conducted with a different perspective, compared to existing survey papers that mostly focus on just video and image deepfakes. This survey mainly focuses on audio deepfakes that are overlooked in most of the existing surveys. This article's most important contribution is to critically analyze and provide a unique source of audio deepfake research, mostly ranging from 2016 to 2021. To the best of our knowledge, this is the first survey focusing on audio deepfakes generation and detection in English.https://www.frontiersin.org/articles/10.3389/fdata.2022.1001063/fullaudio deepfakespoofed audiospoof detectiondeepfake detectiondeepfake generationmisinformation |
| spellingShingle | Zahra Khanjani Gabrielle Watson Vandana P. Janeja Audio deepfakes: A survey Frontiers in Big Data audio deepfake spoofed audio spoof detection deepfake detection deepfake generation misinformation |
| title | Audio deepfakes: A survey |
| title_full | Audio deepfakes: A survey |
| title_fullStr | Audio deepfakes: A survey |
| title_full_unstemmed | Audio deepfakes: A survey |
| title_short | Audio deepfakes: A survey |
| title_sort | audio deepfakes a survey |
| topic | audio deepfake spoofed audio spoof detection deepfake detection deepfake generation misinformation |
| url | https://www.frontiersin.org/articles/10.3389/fdata.2022.1001063/full |
| work_keys_str_mv | AT zahrakhanjani audiodeepfakesasurvey AT gabriellewatson audiodeepfakesasurvey AT vandanapjaneja audiodeepfakesasurvey |