Orpheus TTS — Expressive Text-to-Speech

OpenAI-compatible text-to-speech with emotion control. Drop-in replacement for OpenAI's TTS API — change base_url and your existing openai.audio.speech.create() code works unchanged. Supports expressive tags like <laugh>, <chuckle>, <sigh>, and <gasp>.

GPU: 1x A10G · Cold start: 8-15 min (first boot downloads 4 GB model) · OpenAI-compatible: Yes · No API keys required

Prerequisites

• Convox rack v3.24.6+ with GPU-capable nodes (g5.xlarge recommended)
• No HuggingFace token required — model weights are public

Security Note

The Orpheus-FastAPI upstream exposes configuration endpoints (/save_config, /restart_server, /get_config) without authentication. For production use, add authentication via a reverse proxy or Convox's API key support.

Five-Minute Quickstart

git clone https://github.com/convox-examples/inference-examples.git
cd inference-examples/orpheus-tts

convox apps create orpheus-tts
convox deploy -a orpheus-tts

No environment variables needed for the default configuration. First deploy takes 8-15 minutes as the 4 GB model downloads on first boot.

Architecture

Two services work together — only the inference backend needs a GPU:

Service	Role	Port	GPU
api	Orpheus-FastAPI — handles TTS requests, SNAC audio decoding	5005 (public)	No (CPU)
llama	llama.cpp server — runs the 3B language model (Q8 quantization)	5006 (internal)	Yes

The api service receives OpenAI-compatible TTS requests, sends text to the llama service for token generation, then decodes the tokens into audio using the SNAC codec on CPU.

Get the Endpoint

convox services -a orpheus-tts

SERVICE  DOMAIN                                         PORTS
api      api.orpheus-tts.org-abc123.convox.cloud         443:5005

Test It

curl:

ENDPOINT=$(convox services -a orpheus-tts | awk '$1 == "api" {print $2}')

curl -s "https://$ENDPOINT/v1/audio/speech" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Hello! <laugh> This is Orpheus speaking with emotion.",
    "model": "orpheus",
    "voice": "tara",
    "response_format": "wav"
  }' -o output.wav

Python (OpenAI SDK):

from openai import OpenAI

client = OpenAI(
    base_url="https://api.orpheus-tts.org-abc123.convox.cloud/v1",
    api_key="not-needed",
)

response = client.audio.speech.create(
    model="orpheus",
    voice="tara",
    input="Welcome to Convox! <sigh> That was a long deploy.",
)
response.stream_to_file("output.wav")

Voices

Voice	Gender	Language
tara (default)	Female	English
leah	Female	English
jess	Female	English
mia	Female	English
zoe	Female	English
leo	Male	English
dan	Male	English
zac	Male	English

Additional voices for French, German, Spanish, Italian, Korean, Hindi, and Mandarin are available with language-specific model variants.

Emotion Tags

Insert tags directly in the input text:

Tag	Effect
<laugh>	Laughter
<chuckle>	Light chuckle
<sigh>	Sighing
<gasp>	Gasping
<cough>	Coughing
<sniffle>	Sniffling
<groan>	Groaning
<yawn>	Yawning

Example: "I can't believe it! <gasp> That's amazing <laugh>"

AWS Instance Sizing

Instance	GPU	VRAM	Notes
g5.xlarge	1x A10G	24 GB	Recommended — 3B Q8 model uses ~5-6 GB VRAM
g5.2xlarge	1x A10G	24 GB	More CPU for concurrent audio encoding

Cost and Break-Even vs. OpenAI

OpenAI's hosted tts-1 charges $15/1M characters. Self-hosted on g5.xlarge (~$1.01/hr, ~$735/mo with min: 1):

Monthly Characters	OpenAI Cost	Self-Hosted	Savings
5M	$75	~$735	-$660 (use OpenAI)
50M	$750	~$735	~$15 (break-even)
100M	$1,500	~$735	$765 (51%)
500M	$7,500	~$735	$6,765 (90%)

Break-even is ~49M characters/month. Below that, OpenAI's hosted API is cheaper. The real value is features OpenAI doesn't offer: emotion tags, audio data privacy, and no per-character metering at high volume.

To reduce idle cost, set min: 0 on the llama service for scale-to-zero (adds 8-15 min cold start on first request).

Configuration

Environment variables for the api service:

Variable	Default	Notes
ORPHEUS_API_TIMEOUT	120	Seconds; increase for long-form audio
ORPHEUS_MAX_TOKENS	8192	Increase for very long text
ORPHEUS_TEMPERATURE	0.6	Higher = more expressive
ORPHEUS_TOP_P	0.9	Sampling parameter

convox env set ORPHEUS_TEMPERATURE=0.8 -a orpheus-tts

Budget Controls

convox budget set orpheus-tts --monthly-cap-usd 200 --at-cap-action alert-only
convox cost -a orpheus-tts

GPU Observability

Enable GPU telemetry in Rack Settings to surface per-app GPU utilization in the Console GPU Dashboard.

Troubleshooting

Symptom	Cause	Fix
Long first deploy (15+ min)	Model download (4 GB) on first boot	Normal; subsequent restarts are faster
Connection refused from api	llama.cpp server still loading	Wait for llama service health check
Audio quality issues	Temperature too high/low	Adjust ORPHEUS_TEMPERATURE (default 0.6 is recommended)
Timeout on long text	Text exceeds token limit	Increase ORPHEUS_MAX_TOKENS and ORPHEUS_API_TIMEOUT
OOM errors	Model too large for available VRAM	Use Q4_K_M quantization variant (change MODEL_URL in Dockerfile.llama)