Chatterbox version?

[BonneAppetit]

Hi!

Thanks for all the hard work with porting python models to a more efficient cpp!

The chatterbox model that you have provided its V1 or V2 not the V3 multillingual version right? Is there a possibility for an update since the newer version claims to have more languages and less hallucinations?

And also, having more clear benchmarks which are constantly and consistently updated could be really good because manual testing is often confusing for those who are new to the field.

Given your expertise, for a very quick STT and TTS latency and low error rate, what would be the best pair? And if we include voice cloning capabilities for the TTS, does your answer change?

Thanks

[0xShug0]

@BonneAppetit Thanks! Audio.cpp supports Chatterbox v3 but the default is v2 following the upstream example. The GGUFs also packed the v3 weights, but I forgot to add the v2/v3 switch in the cli...Just updated the code and you can check the latest commit.

audiocpp_cli \
  --task clon \
  --family chatterbox \
  --model /path/to/Chatterbox-GGUF/chatterbox-q8_0.gguf \
  --backend cuda \
  --language sw \
  --text "Habari, huu ni mtihani mfupi wa sauti." \
  --voice-ref /path/to/reference.wav \
  --session-option chatterbox.multilingual_t3=v3 \
  --out out.wav

On benchmarks: agreed. There is a PR to improve the benchmarking #91. I'd prefer to leave this part community-driven rather than have it become my main focus.

For STT + TTS: I definitely recommend Nemotron ASR for speech recognition. It's fast and accurate (check the benchmark here https://github.com/Martossien/transcria/blob/main/docs/STT_BENCHMARK_REAL_MEETINGS.md). For TTS, if you don't care about voice, I'd suggest a CPU-friendly small TTS model or Supertonic 3 in audio.cpp. If you do care about voice cloning, I'd recommend Omnivoice.

[BonneAppetit]

Thanks for the reply! Although I'm now quite confused, for example Omnivoice here with a gguf reports to have a similar RTF as the original (confirmed also in the #91 . And those are GGUF. However there's a different low-star repo that claims GGUF do a 3x speedup https://github.com/newgrit1004/omnivoice-triton. Any ideas? Sorry for the possible confusion, just trying to wrap my head around things as I have no idea how this CUDA graph capture works

[0xShug0]

@BonneAppetit The performance boost from that repo comes from its optimizations and use of a different kernel. GGUF is just a model format. The actual performance gains come from the quantization format, such as Q8 or Q4, together with optimized kernel support.

[0xShug0]

Check the test here https://www.reddit.com/r/LocalLLM/comments/1v4ztd2/audiocpp_release_04_higgs_audio_v3_tts_4b_10x/