Repository: blue-fish/Real-Time-Voice-Cloning (v1.0)
Description: RTVC-FT-1 takes the RTVC-7 model and trains it on data from each of the 6 target speakers. Each speaker has its own finetuned model.
Click here for more voice cloning experiments.
| Name | Model | Steps | Batch Size | Datasets Used | Speakers | Audio Duration | |
|---|---|---|---|---|---|---|---|
| Speaker Encoder: | Pretrained | GE2E | 1,564,501 | 64 | LibriSpeech train-other-500 VoxCeleb1 Dev A-D VoxCeleb2 Dev A-H |
8371 | 3201 hours |
| Synthesizer: |
LS_Taco1_FT_296k (1 model/speaker) |
Tacotron 1 | 285,000 | 12 | LibriSpeech train-clean-100 LibriSpeech train-clean-360 |
1172 | 436 hours |
| 10,000 | 12 | LibriTTS train-clean-100 LibriTTS train-clean-360 |
1172 | 245 hours | |||
| 1,000 | 12 | VCTK p240 | 1 | 0.33 hours | |||
| VCTK p260 | 1 | 0.30 hours | |||||
| LibriSpeech test-clean/1320 | 1 | 0.13 hours | |||||
| LibriSpeech test-clean/3575 | 1 | 0.13 hours | |||||
| LibriSpeech test-clean/6829 | 1 | 0.14 hours | |||||
| LibriSpeech test-clean/8230 | 1 | 0.14 hours | |||||
| Vocoder: | LS_GTA_1159k | WaveRNN | 1,159,000 | 50 | LibriSpeech train-clean-100 LibriSpeech train-clean-360 |
1172 | 436 hours |
All speakers are unseen during training for Google and RTVC-7 only. The first row is the reference audio used to compute the speaker embedding. The rows below that are synthesized using that speaker embedding.
| VCTK p240 | VCTK p260 | LibriSpeech 1320 | LibriSpeech 3575 | LibriSpeech 6829 | LibriSpeech 8230 | |
|---|---|---|---|---|---|---|
| Reference: | ||||||
| Synthesized: | ||||||
| 0: Take a look at these pages for crooked creek drive. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||
| 1: There are several listings for gas station. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||
| 2: Here's the forecast for the next four days. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||
| 3: Here is some information about the Gospel of John. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||
| 4: His motives were more pragmatic and political. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||
| 5: She had three brothers and two sisters. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||
| 6: This work reflects a quest for lost identity, a recuperation of an unknown past. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||
| 7: There were many editions of these works still being used in the nineteenth century. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||
| 8: Modern birds are classified as coelurosaurs by nearly all palaeontologists. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||
| 9: He was being fitted for ruling the state, in the words of his biographer. | ||||||
| Google: | ||||||
| RTVC-7: | ||||||
| RTVC-FT-1: | ||||||