Enhancing Voice Cloning Quality through Data Selection and Alignment-Based Metrics
Voice cloning, an emerging field in the speech-processing area, aims to generate synthetic utterances that closely resemble the voices of specific individuals. In this study, we investigated the impact of various techniques on improving the quality of voice cloning, specifically focusing on a low-qu...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/14/8049 |
| _version_ | 1797590458609696768 |
|---|---|
| author | Ander González-Docasal Aitor Álvarez |
| author_facet | Ander González-Docasal Aitor Álvarez |
| author_sort | Ander González-Docasal |
| collection | DOAJ |
| description | Voice cloning, an emerging field in the speech-processing area, aims to generate synthetic utterances that closely resemble the voices of specific individuals. In this study, we investigated the impact of various techniques on improving the quality of voice cloning, specifically focusing on a low-quality dataset. To contrast our findings, we also used two high-quality corpora for comparative analysis. We conducted exhaustive evaluations of the quality of the gathered corpora in order to select the most-suitable data for the training of a voice-cloning system. Following these measurements, we conducted a series of ablations by removing audio files with a lower signal-to-noise ratio and higher variability in utterance speed from the corpora in order to decrease their heterogeneity. Furthermore, we introduced a novel algorithm that calculates the fraction of aligned input characters by exploiting the attention matrix of the Tacotron 2 text-to-speech system. This algorithm provides a valuable metric for evaluating the alignment quality during the voice-cloning process. We present the results of our experiments, demonstrating that the performed ablations significantly increased the quality of synthesised audio for the challenging low-quality corpus. Notably, our findings indicated that models trained on a 3 h corpus from a pre-trained model exhibit comparable audio quality to models trained from scratch using significantly larger amounts of data. |
| first_indexed | 2024-03-11T01:20:47Z |
| format | Article |
| id | doaj.art-8ffb8df7ef294f9da86de57304238150 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T01:20:47Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-8ffb8df7ef294f9da86de573042381502023-11-18T18:07:28ZengMDPI AGApplied Sciences2076-34172023-07-011314804910.3390/app13148049Enhancing Voice Cloning Quality through Data Selection and Alignment-Based MetricsAnder González-Docasal0Aitor Álvarez1Fundación Vicomtech, Basque Research and Technology Alliance (BRTA), 20009 Donostia-San Sebastián, SpainFundación Vicomtech, Basque Research and Technology Alliance (BRTA), 20009 Donostia-San Sebastián, SpainVoice cloning, an emerging field in the speech-processing area, aims to generate synthetic utterances that closely resemble the voices of specific individuals. In this study, we investigated the impact of various techniques on improving the quality of voice cloning, specifically focusing on a low-quality dataset. To contrast our findings, we also used two high-quality corpora for comparative analysis. We conducted exhaustive evaluations of the quality of the gathered corpora in order to select the most-suitable data for the training of a voice-cloning system. Following these measurements, we conducted a series of ablations by removing audio files with a lower signal-to-noise ratio and higher variability in utterance speed from the corpora in order to decrease their heterogeneity. Furthermore, we introduced a novel algorithm that calculates the fraction of aligned input characters by exploiting the attention matrix of the Tacotron 2 text-to-speech system. This algorithm provides a valuable metric for evaluating the alignment quality during the voice-cloning process. We present the results of our experiments, demonstrating that the performed ablations significantly increased the quality of synthesised audio for the challenging low-quality corpus. Notably, our findings indicated that models trained on a 3 h corpus from a pre-trained model exhibit comparable audio quality to models trained from scratch using significantly larger amounts of data.https://www.mdpi.com/2076-3417/13/14/8049voice cloningspeech synthesisspeech quality evaluation |
| spellingShingle | Ander González-Docasal Aitor Álvarez Enhancing Voice Cloning Quality through Data Selection and Alignment-Based Metrics Applied Sciences voice cloning speech synthesis speech quality evaluation |
| title | Enhancing Voice Cloning Quality through Data Selection and Alignment-Based Metrics |
| title_full | Enhancing Voice Cloning Quality through Data Selection and Alignment-Based Metrics |
| title_fullStr | Enhancing Voice Cloning Quality through Data Selection and Alignment-Based Metrics |
| title_full_unstemmed | Enhancing Voice Cloning Quality through Data Selection and Alignment-Based Metrics |
| title_short | Enhancing Voice Cloning Quality through Data Selection and Alignment-Based Metrics |
| title_sort | enhancing voice cloning quality through data selection and alignment based metrics |
| topic | voice cloning speech synthesis speech quality evaluation |
| url | https://www.mdpi.com/2076-3417/13/14/8049 |
| work_keys_str_mv | AT andergonzalezdocasal enhancingvoicecloningqualitythroughdataselectionandalignmentbasedmetrics AT aitoralvarez enhancingvoicecloningqualitythroughdataselectionandalignmentbasedmetrics |