Skip to content

feat(chat): openai-chat 协议按 (provider, upstream-model) 维度透传供应商私有 JSON 字段(extra_body)#52

Open
yangsijie666 wants to merge 2 commits into
ZhiYi-R:mainfrom
yangsijie666:feat/openai-chat-offer-extra-body
Open

feat(chat): openai-chat 协议按 (provider, upstream-model) 维度透传供应商私有 JSON 字段(extra_body)#52
yangsijie666 wants to merge 2 commits into
ZhiYi-R:mainfrom
yangsijie666:feat/openai-chat-offer-extra-body

Conversation

@yangsijie666

Copy link
Copy Markdown
Contributor

问题

MoonBridge 在把请求转发到 openai-chat 协议的上游时,没有任何路径让用户传递不属于 OpenAI Chat Completions 标准字段的顶层 JSON 字段。但很多兼容 openai-chat 接口的供应商都依赖这种私有开关,例如 DashScope 兼容接口上控制是否启用联网搜索的 enable_search。这些信号只能在客户端直连供应商时才传得到;一旦经过 MoonBridge,相关字段会被丢弃,对应能力无法启用。

承载粒度方面也有真实差异:

  • 同一个 provider 下不同的 upstream model 对供应商私有开关的需求并不相同。是否启用搜索、是否启用某项实验特性,往往是逐 model 决定的,不应在一个 provider 下被所有模型共享。
  • 同一个 upstream model 名(例如 qwen3-plus)可以被多个 provider 同时提供(DashScope 与百炼都暴露这个模型),而不同 provider 的兼容接口对这些私有字段的支持情况并不一致。

因此正确的承载粒度是 (provider, upstream-model) 维度——也就是 ModelMeta 这一层,和 WebSearchExtensions 等已有字段同级。

复现场景

我使用的配置:

mode: "Transform"

server:
  addr: "127.0.0.1:38440"

extensions:
  visual:
    enabled: true
    config:
      provider: "aliyun"
      model: "qwen3.6-plus"
      max_rounds: 10
      max_tokens: 10000

models:
  deepseek-v4-pro:
    context_window: 1000000
    max_output_tokens: 384000
    extensions:
      visual:
        enabled: true
    default_reasoning_level: "high"
    supported_reasoning_levels:
      - effort: "high"
        description: "High reasoning effort"
      - effort: "max"
        description: "Extra high reasoning effort"
  qwen3.6-plus:
    context_window: 1000000
    max_output_tokens: 64000
    default_reasoning_level: "high"
    supported_reasoning_levels:
      - effort: "high"
        description: "High reasoning effort"

providers:
  aliyun:
    base_url: "https://dashscope.aliyuncs.com/compatible-mode/v1"
    api_key: "sk-***"
    protocol: openai-chat
    offers:
      - model: deepseek-v4-pro
      - model: qwen3.6-plus

routes:
  moonbridge:
    model: deepseek-v4-pro
    provider: aliyun

defaults:
  model: moonbridge
  max_tokens: 65536

目标:在不改动 OpenAI Chat Completions 标准字段定义的前提下,让 deepseek-v4-pro 透传 enable_search: true 给 DashScope,让 qwen3.6-plus 不带任何私有字段。

修改前:MoonBridge 没有提供任何方式达到这一点。

设计

YAML 入口

复用项目已有的 providers.<key>.offers[].overrides 机制(同一位置已经承载 web_searchcontext_window 等 per-(provider, model) 覆盖项)。新增 overrides.extra_body 字段:

providers:
  aliyun:
    base_url: "https://dashscope.aliyuncs.com/compatible-mode/v1"
    api_key: "sk-***"
    protocol: openai-chat
    offers:
      - model: deepseek-v4-pro
        overrides:
          extra_body:
            enable_search: true
      - model: qwen3.6-plus
        # 不写 overrides.extra_body 即不注入任何私有字段

extra_body 必须显式声明在某条 offer 下,无隐式继承。同名 model 在不同 provider 下的覆盖完全独立。

出口注入逻辑

所有从 MoonBridge 发出的 openai-chat 请求——非流式直连、流式、visual orchestrator 编排的每一轮——都经过同一个出口函数 ChatProviderAdapter.FromCoreRequest。因此 extra_body 在整条链路上只在一处写、一处读

sequenceDiagram
  participant D as dispatch
  participant CR as CoreRequest
  participant A as ChatProviderAdapter
  participant CHR as ChatRequest
  participant M as MarshalJSON
  participant U as Upstream

  D->>D: 路由解析<br/>得到 preferred.ProviderKey 与 preferred.UpstreamModel
  D->>CR: Extensions["openai_chat"]["extra_body"]<br/>= ModelMeta.ExtraBody
  Note over D,CR: 整个请求生命周期内只写一次

  rect rgb(240, 240, 240)
    Note over A,U: 直连、流式、visual orchestrator 每一轮均走此路径
    A->>CR: FromCoreRequest(coreReq)
    A->>CHR: chatReq.ExtraParams = extractChatExtraBody(coreReq.Extensions)
    A->>M: json.Marshal(chatReq)
    M->>M: 把 ExtraParams 平铺到顶层 JSON<br/>标准字段优先,碰撞键被丢弃
    M->>U: POST /v1/chat/completions<br/>顶层含 enable_search 等开关
  end
Loading

这与仓库里已经成形的 Extensions["openai"]["reasoning"]["effort"](用于跨协议传递 reasoning_effort 信号)同源——新协议元信息通过 CoreRequest.Extensions 携带、由目标协议的适配器消费——extra_body 沿用一致的约定。

字段碰撞保护

ChatRequest.MarshalJSON 先按结构体生成标准字段的 JSON 输出,再把 ExtraParams不与已有键冲突的键合并进去。也就是说:

  • 即便 ExtraParams 里写了 "model": "evil-override",最终出口 JSON 的 model 字段仍然是 chatReq.Model,不会被覆盖。
  • 其余不与标准字段重名的供应商私有字段正常出现在顶层。

测试覆盖

go test ./... 全部通过。新增测试覆盖:

配置层internal/config/config_test.go

  • TestLoadFromYAMLParsesOfferExtraBody:把含 providers.X.offers[].overrides.extra_body 的 YAML 喂给 loader,断言:
    • 同一 provider 下两条 offer 各自独立的 ExtraBody,互不污染
    • 同一 upstream model 名被两个 provider 提供时各自独立
    • 不写 overrides.extra_body 时 ModelMeta.ExtraBody 为空

适配器层internal/protocol/chat/chat_test.go

  • TestAdapter_ChatExtraBody_FromExtensions:CoreRequest.Extensions 中的 extra_body 被翻译到 ChatRequest.ExtraParams
  • TestAdapter_ChatExtraBody_AbsentExtensionLeavesExtraParamsNil:Extensions 缺失时 ExtraParams 留空。

DTO 层internal/protocol/chat/chat_test.go

  • 4 条 TestTypes_ChatRequest_ExtraParams_* 覆盖 MarshalJSON 平铺、空值、键冲突保护、多键。

端到端集成internal/protocol/chat/chat_test.go,用 httptest.NewServer 拦截 chat 出口请求体,无任何真实上游调用)

  • TestOpenAIChat_ExtraBody_PropagatesOnNonStreamDirect:非流式直连路径——出口 JSON 顶层含配置的供应商私有字段,model/messages 未被覆盖。
  • TestOpenAIChat_ExtraBody_PropagatesOnStreaming:流式路径——出口 JSON 同时含 stream: truestream_options 与供应商私有字段。
  • TestOpenAIChat_ExtraBody_PropagatesThroughVisualOrchestrator:visual orchestrator 编排路径——逐轮断言 orchestrator 的每一次上游调用都带上供应商私有字段。

…-chat

RED state. Two layers of tests are added; both fail to compile because the
target fields do not exist yet:

- internal/config/config_test.go: TestLoadFromYAMLParsesOfferExtraBody asserts
  that providers.<key>.offers[].overrides.extra_body is propagated to the
  matching ModelMeta in cfg.ProviderDefs[provider].Models[upstream], with
  per-(provider, upstream-model) isolation. Same model name offered by two
  providers must keep separate ExtraBody maps; offers without overrides must
  observe an empty ExtraBody.

- internal/protocol/chat/chat_test.go: four cases for the DTO-layer mechanism
  TestTypes_ChatRequest_ExtraParams_*. They verify that ChatRequest.ExtraParams
  is flattened into the top-level JSON object via MarshalJSON, that nil leaves
  the body unchanged, that existing fields cannot be clobbered, and that
  multiple keys round-trip cleanly.

This is the intended RED state for a TDD GREEN follow-up that introduces
ModelMeta.ExtraBody and ChatRequest.ExtraParams.
Lets users declare vendor-specific top-level JSON switches (such as
DashScope's enable_search) scoped to a single (provider, upstream-model)
tuple. Switches are merged into the outbound Chat Completions request
body only when the offer that resolves a request configures them;
sibling offers on the same provider are unaffected.

Design rationale: provider-level scope is too coarse — the same provider
typically serves multiple distinct models with different switch needs, and
the same model name offered by different providers may require different
switches. Attaching extra_body to ModelMeta (per (provider, upstream-model))
matches the actual granularity of vendor requirements.

YAML surface (only path; no top-level models.<name>.extra_body):

  providers:
    aliyun:
      protocol: openai-chat
      offers:
        - model: qwen3-plus
          overrides:
            extra_body:
              enable_search: true
        - model: deepseek-v4-pro
          # no overrides.extra_body → no vendor switches injected

Wiring:
- ChatRequest gains ExtraParams plus a custom MarshalJSON that flattens
  the map to the top-level JSON object. Standard fields take precedence so
  model/messages/stream cannot be clobbered by misconfiguration.
- ModelDef and ModelMeta gain ExtraBody fields. The loader's existing
  offer-override pipeline (mergeModelDefOverrides + applyModelOverrides)
  carries the map from file config to ModelMeta. convert.go round-trips
  ExtraBody through ModelDefFileConfig for the management API.
- The dispatcher writes the resolved value once into
  CoreRequest.Extensions["openai_chat"]["extra_body"] after route
  resolution. ChatProviderAdapter.FromCoreRequest reads it on every
  outbound chat completion call, so direct, streaming, and per-round
  visual-orchestrator paths all observe the same vendor switches without
  protocol-specific call-site duplication. This mirrors the existing
  Extensions["openai"]["reasoning"]["effort"] convention.
- docs/CONFIGURATION.md and config.example.yml document the new path.

End-to-end coverage is captured by three httptest-based integration tests
that intercept the outbound chat completion body without any real upstream
call: TestOpenAIChat_ExtraBody_PropagatesOnNonStreamDirect,
TestOpenAIChat_ExtraBody_PropagatesOnStreaming, and
TestOpenAIChat_ExtraBody_PropagatesThroughVisualOrchestrator.

The RED test (parent commit) compiles and passes after this change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant