Skip to content

Fix: inference-parser misses /chat/completions from /v1-less proxies (opencode, litellm) #823

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
huang195 merged 1 commit into from
Aug 31, 2026
+45 −1
Merged

Fix: inference-parser misses /chat/completions from /v1-less proxies (opencode, litellm) #823

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion authbridge/authlib/plugins/inferenceparser/plugin.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ func (p *InferenceParser) OnRequest(_ context.Context, pctx *pipeline.Context) p
// "inference-parser is in this pipeline" from config, not per-event rows.
var ext *pipeline.InferenceExtension
switch endpointPath(pctx) {
case "/v1/chat/completions", "/v1/completions":
case "/v1/chat/completions", "/v1/completions", "/chat/completions", "/completions":

@mrsabath mrsabath Aug 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): same-class gap on the Anthropic side. anthropicMessagesPath is /v1/messages, so a /v1-stripping proxy (this PR's exact scenario) posting Anthropic-style traffic to /messages would still fall through to the default arm and record no inference telemetry. Out of scope for #822, but worth a follow-up (or a matching /messages case here) so the asymmetry is a deliberate choice rather than an oversight.

ext = parseOpenAIRequest(pctx.Body)
case anthropicMessagesPath:
ext = parseAnthropicRequest(pctx.Body)
Expand Down
44 changes: 44 additions & 0 deletions authbridge/authlib/plugins/inferenceparser/plugin_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,50 @@ func TestInferenceParser_OnResponse_CapturesToolCalls(t *testing.T) {
}
}

// TestInferenceParser_VlessPath covers OpenAI-compatible proxies (e.g. opencode via
// litellm) that strip the /v1 prefix and post directly to /chat/completions or
// /completions. Without these path variants the parser falls to the default arm
// and records no inference telemetry.
func TestInferenceParser_VlessPath_ChatCompletions(t *testing.T) {
p := NewInferenceParser()
pctx := &pipeline.Context{
Path: "/chat/completions",
Body: []byte(`{"model":"gpt-4","messages":[{"role":"user","content":"hi"}],"stream":false}`),
}
action := p.OnRequest(context.Background(), pctx)
if action.Type != pipeline.Continue {
t.Fatalf("expected Continue, got %v", action.Type)
}
ext := pctx.Extensions.Inference
if ext == nil {
t.Fatal("Extensions.Inference is nil for /chat/completions")
}
if ext.Model != "gpt-4" {
t.Errorf("Model = %q, want gpt-4", ext.Model)
}
if !ext.IsAction {
t.Error("IsAction should be true")
}
}

func TestInferenceParser_VlessPath_Completions(t *testing.T) {
p := NewInferenceParser()
pctx := &pipeline.Context{
Path: "/completions",
Body: []byte(`{"model":"codellama","messages":[{"role":"user","content":"hi"}]}`),
}
action := p.OnRequest(context.Background(), pctx)
if action.Type != pipeline.Continue {
t.Fatalf("expected Continue, got %v", action.Type)
}
if pctx.Extensions.Inference == nil {
t.Fatal("Extensions.Inference is nil for /completions")
}
if pctx.Extensions.Inference.Model != "codellama" {
t.Errorf("Model = %q, want codellama", pctx.Extensions.Inference.Model)
}
}

func TestInferenceParser_NonMatchingPath(t *testing.T) {
p := NewInferenceParser()
pctx := &pipeline.Context{
Expand Down
Loading