I'll start by saying that I've tried many TTS models for voice cloning and dubbing.
I've encountered this common problem: when dubbing a sentence, the last word spoken is often cut off, or the speech stops before the sentence ends.
I'm sure TTS models are excellent, because I've also tried them to generate simple text-to-speech. So the model itself works well.
The problem is text-to-speech during dubbing. More stringent rules should be set, such as requiring it to finish all words correctly and not cut them off. Furthermore, for example, you could leave some leeway in the timing of sentences, so that it has a safety margin to finish words and sentences.
I don't know if this is feasible, but I'm noticing that it's a shame to have all these TTS models that clone excellently, and then end up with incorrect results. Even re-dubbing often fails to produce a correct and decently pronounced sentence.
I therefore believe that, with current TTS models, the level is already very high, and there's no point in adding new ones if we don't focus on and improve the final result.
I therefore ask that we move on to a careful review of the final result, improving the speech processing. Now, in my opinion, is the time to focus. I'm sure that improving this phase will improve the processing across all the various TTS models.
I would also recommend improving the speech speed: I've often noticed that the speaker speaks slowly, and as a result, there's not enough time to pronounce all the words in the sentence.
In fact, I would even consider setting a default speed-up rule to ensure that words aren't cut off (I don't know, it's just an idea).
Unfortunately, the result isn't usable today.
I'll start by saying that I've tried many TTS models for voice cloning and dubbing.
I've encountered this common problem: when dubbing a sentence, the last word spoken is often cut off, or the speech stops before the sentence ends.
I'm sure TTS models are excellent, because I've also tried them to generate simple text-to-speech. So the model itself works well.
The problem is text-to-speech during dubbing. More stringent rules should be set, such as requiring it to finish all words correctly and not cut them off. Furthermore, for example, you could leave some leeway in the timing of sentences, so that it has a safety margin to finish words and sentences.
I don't know if this is feasible, but I'm noticing that it's a shame to have all these TTS models that clone excellently, and then end up with incorrect results. Even re-dubbing often fails to produce a correct and decently pronounced sentence.
I therefore believe that, with current TTS models, the level is already very high, and there's no point in adding new ones if we don't focus on and improve the final result.
I therefore ask that we move on to a careful review of the final result, improving the speech processing. Now, in my opinion, is the time to focus. I'm sure that improving this phase will improve the processing across all the various TTS models.
I would also recommend improving the speech speed: I've often noticed that the speaker speaks slowly, and as a result, there's not enough time to pronounce all the words in the sentence.
In fact, I would even consider setting a default speed-up rule to ensure that words aren't cut off (I don't know, it's just an idea).
Unfortunately, the result isn't usable today.