Skip to content

Allow a custom base URL for OpenAI integrations - #6

Open
aldinsmajlovic wants to merge 1 commit into
kyvaith:mainfrom
aldinsmajlovic:openai-base-url
Open

aldinsmajlovic wants to merge 1 commit into
kyvaith:mainfrom
aldinsmajlovic:openai-base-url

Conversation

@aldinsmajlovic

Copy link
Copy Markdown

The OpenAI (realtime) and OpenAI Cloud integrations get an optional Base URL field. Blank keeps today's behavior (api.openai.com). When it's set, these go to that server instead:

  • realtime speech-to-speech (OpenAIRealtimeLLMService)
  • composed-pipeline STT (OpenAIRealtimeSTTService) and TTS (OpenAITTSService)
  • the Assist bridge STT and TTS, including the streaming transcription WebSocket
  • the model list in the integration editor. It's unfiltered for custom servers, since they name models freely.

This lets a pipeline run against self-hosted OpenAI-compatible servers. Examples are an OpenAI Realtime-compatible speech-to-speech server such as huggingface/speech-to-speech, or local STT/TTS servers such as Speaches or Chatterbox. That gives fully local voice with barge-in, which the Local runtime bridge can't provide.

Details:

  • The realtime WebSocket URL comes from the base URL: http://host:8000/v1 becomes ws://host:8000/v1/realtime, https becomes wss, and a URL that already ends in /realtime is kept as-is.
  • With a base URL, the API key is optional. A placeholder is sent if it's blank. The add-on's global openai_api_key is never sent to a custom server.
  • Add a second OpenAI integration from Integrations to keep one pointing at OpenAI and one at your own server.
  • All three Pipecat services already accept base_url in the pinned Pipecat 1.4 (checked).

Testing

  • tests/test_openai_base_url.py: URL mapping, key handling, and that other kinds ignore base_url. The full suite passes (python -m unittest discover -s tests, 32 tests).
  • Assist bridge TTS and composed TTS built against a local Chatterbox server that speaks the OpenAI speech protocol. It returned WAV audio.
  • End to end against huggingface/speech-to-speech serve (faster-whisper small.en on CPU, a local llama.cpp Qwen model over chat completions, a local OpenAI-compatible TTS). I built the service with this PR's _openai_realtime_service on the pinned Pipecat 1.4, using the default flow with a local OpenAI integration (base_url=http://127.0.0.1:8765/v1, no API key). Then I streamed two spoken utterances into it in real time: a request for a long story, and an interruption ("Stop. Never mind the story. What is two plus two?") 3 s into the answer. With the flow's Interrupt response enabled, the story was cut off about 1.1 s into the interruption and the assistant answered "2 plus 2 is 4.", with first audio about 2.3 s after the question ended.
  • With the default interrupt_response: false, the server correctly ignores speech during a response (it logs interrupt_response disabled, ignoring). The answer to the interruption then queues behind the rest of the story. So local barge-in needs that flow toggle on, and the docs paragraph says so.

The UI bundle (app/ui/index.js) is rebuilt from ui-src. index.html is left at its current version stamp.

🤖 Generated with Claude Code

The OpenAI (realtime) and OpenAI Cloud integrations get an optional Base
URL. Blank keeps api.openai.com. When set, realtime speech-to-speech,
realtime STT, TTS, the Assist bridge STT/TTS and the model list go to that
server instead, so a self-hosted OpenAI Realtime-compatible server (or any
OpenAI-compatible STT/TTS server) can back a pipeline. The realtime
WebSocket URL is derived from the base URL. The API key is optional with a
base URL, and the global OpenAI key is never sent to a custom server.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Aldin <smajlovicaldin52@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant