chatterbox-multi

A Chatterbox model by Resemble AI, from ResembleAI/chatterbox, packaged for phoonnx, our ONNX text-to-speech library.

Model details

Architecture	Chatterbox
Upstream	ResembleAI/chatterbox
ONNX conversion	onnx-community/chatterbox-multilingual-ONNX
Author	Resemble AI
Languages	ar, bg-BG, cs-CZ, da-DK, de-DE, el-GR, en-US, es-ES, fi-FI, fr-FR, he-IL, hi-IN, hu-HU, it-IT, ja-JP, ko-KR, ms-MY, nl-NL, no-NO, pl-PL, pt-PT, ro-RO, ru-RU, sk-SK, sv-SE, sw, ta-IN, tr-TR, vi-VN, zh-CN
Text input	Unicode codepoints of the text

How to use

pip install --pre phoonnx

import wave
from phoonnx.model_manager import TTSModelManager

manager = TTSModelManager()
manager.load()
manager.merge_default_voices()

voice = manager.voices["chatterbox/multilingual/ar"].load()

with wave.open("output.wav", "wb") as wav_file:
    text = "..."  # a sentence in ar
    voice.synthesize_wav(text, wav_file)

The voice clones a speaker from reference audio; the phoonnx README shows how to pass it.

In OVOS, phoonnx installs the ovos-tts-plugin-phoonnx plugin. Select the voice in mycroft.conf:

{"tts": {"module": "ovos-tts-plugin-phoonnx", "ovos-tts-plugin-phoonnx": {"voice": "chatterbox/multilingual/ar"}}}

Text front end

phoonnx hands the text to scriptconv, our text conversion library, which normalises it and turns it into the model's input:

• unicode: UnicodeCodepointPhonemizer from scriptconv.phonemizers.base, in scriptconv itself.

phoonnx voices

• chatterbox/multilingual/ar
• chatterbox/multilingual/bg
• chatterbox/multilingual/cs
• chatterbox/multilingual/da
• chatterbox/multilingual/de
• chatterbox/multilingual/el
• chatterbox/multilingual/en
• chatterbox/multilingual/es
• chatterbox/multilingual/fi
• chatterbox/multilingual/fr
• chatterbox/multilingual/he
• chatterbox/multilingual/hi
• chatterbox/multilingual/hu
• chatterbox/multilingual/it
• chatterbox/multilingual/ja
• chatterbox/multilingual/ko
• chatterbox/multilingual/ms
• chatterbox/multilingual/nl
• chatterbox/multilingual/no
• chatterbox/multilingual/pl
• chatterbox/multilingual/pt
• chatterbox/multilingual/ro
• chatterbox/multilingual/ru
• chatterbox/multilingual/sk
• chatterbox/multilingual/sv
• chatterbox/multilingual/sw
• chatterbox/multilingual/ta
• chatterbox/multilingual/tr
• chatterbox/multilingual/vi
• chatterbox/multilingual/zh

Licence

The upstream declares the licence mit in the model card of ResembleAI/chatterbox.

Related projects

• phoonnx: runs this model, in Python and in OVOS.
• scriptconv: text normalisation, script conversion and phonemizers for phoonnx.
• The Chatterbox collection: every Chatterbox model packaged for phoonnx.