Skip to content

yaml-time: v3 config update - #173

Open
SearchSavior wants to merge 5 commits into
mainfrom
yaml-time
Open

SearchSavior wants to merge 5 commits into
mainfrom
yaml-time

Conversation

@SearchSavior

Copy link
Copy Markdown
Owner

PR is still WIP.

This PR implements a full rewrite of the OpenArc configuration system to support a YAML format to make writing config files easier, and decouple logic from the cli module. There are two goals:

  • Ensure openai api compatability
  • Make more challenging models easier to configure

For example, openai format accepts an extra_body parameter which can nest json- very nice. I used this to support the fuil suite of options for qwen3-tts, and each setting is configurable at request time. Unfortunately this powerful feature was safely hidden away by terrible UX, since none of the useful tools which integrate openai v1/audio/speech support extra_body. Now, you can define ALL configurable setting in a way that locks in a precedence order:

Request body --> yaml config --> engine defaults

In this way, OpenArc config files are now fully declarative, and can support extension to whatever new architectures we want to support, no matter what they require, and reguardless of how the API standards change. For example, now max_tokens, temperature, runtime_config and scheduler_config can be set per model; more interestingly kokoro can now accept voice_blend in the yaml, and is resolved at request time meaning you don't have to set voice for it to work with voice_blend; this is powerful because you get the implementation richness of OpenArc in applications that can leverage the inference we provide.

Another example to illustrate:

  # qwen3_tts_elmo:
  #   load_config:
  #     model_type: qwen3_tts_voice_clone
  #     model_path:
  #     engine: openvino
  #     device: GPU.0
  #   qwen3_tts_voice_clone_config:
  #     ref_text:
  #     x_vector_only:
  #     instruct:
  #   qwen3_tts_config:
  #     max_new_tokens:
  #     do_sample:
  #     top_k:
  #     top_p:
  #     temperature:
  #     repetition_penalty:
  #     subtalker_do_sample:
  #     subtalker_top_k:
  #     subtalker_top_p:
  #     subtalker_temperature:
  #     stream:
  #     stream_chunk_frames:
  #     stream_left_context:

Above you can set the reference audio and reference audio transcription of that audio in the config, so the client only has to send model and input (in the voice clone case). Some of these shouldn't even be changed like stream_left_context, but why not expose it all?

to be continued

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant