Repository: CorentinJ/Real-Time-Voice-Cloning @5425557
Description: The synthesizer in RTVC-3 is trained on a small subset of LibriSpeech that does not contain any pauses in the middle of recordings.
Click here for more voice cloning experiments.
| Name | Model | Steps | Batch Size | Datasets Used | Speakers | Audio Duration | |
|---|---|---|---|---|---|---|---|
| Speaker Encoder: | Pretrained | GE2E | 1,564,501 | 64 | LibriSpeech train-other-500 VoxCeleb1 Dev A-D VoxCeleb2 Dev A-H |
8371 | 3201 hours |
| Synthesizer: | LS_subset_270k | Tacotron 2 | 270,000 | 12 | LibriSpeech train-clean-100 LibriSpeech train-clean-360 |
1172 |
48 hours |
| Vocoder: | LS_GTA_1159k | WaveRNN | 1,159,000 | 50 | LibriSpeech train-clean-100 LibriSpeech train-clean-360 |
1172 | 436 hours |
All speakers are unseen during training. The first row is the reference audio used to compute the speaker embedding. The rows below that are synthesized using that speaker embedding.
| VCTK p240 | VCTK p260 | LibriSpeech 1320 | LibriSpeech 3575 | LibriSpeech 6829 | LibriSpeech 8230 | |
|---|---|---|---|---|---|---|
| Reference: | ||||||
| Synthesized: | ||||||
| 0: Take a look at these pages for crooked creek drive. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||
| 1: There are several listings for gas station. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||
| 2: Here's the forecast for the next four days. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||
| 3: Here is some information about the Gospel of John. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||
| 4: His motives were more pragmatic and political. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||
| 5: She had three brothers and two sisters. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||
| 6: This work reflects a quest for lost identity, a recuperation of an unknown past. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||
| 7: There were many editions of these works still being used in the nineteenth century. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||
| 8: Modern birds are classified as coelurosaurs by nearly all palaeontologists. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||
| 9: He was being fitted for ruling the state, in the words of his biographer. | ||||||
| Google: | ||||||
| RTVC-2: | ||||||
| RTVC-3: | ||||||