Skip to content

fix(billing): align pre-output Claude refusal settlement with upstream no-charge semantics #516

Description

@LIghtJUNction

Upstream

Problem

Anthropic documents that a refusal which happens before any output is not billed upstream: stop_reason=refusal, empty content, and output_tokens=0. The response may still contain non-zero input-token usage for rate-limit/accounting purposes.

LMM currently detects the refusal in apps/api-go/relay/channel/claude/relay-claude.go, but maybeMarkClaudeRefusal only writes ContextKeyAdminRejectReason. The returned usage then reaches apps/api-go/service/text_quota.go; textQuotaSummary.hasBillableUsage() treats non-zero input tokens as billable, so ratio/fixed/tiered settlement can still charge the user.

This is the same underlying gap fixed by upstream #7552.

Why not directly cherry-pick #7552

Do not mechanically copy the upstream patch yet:

  1. #7552 is still open and changes billing policy through a new global opt-in setting.
  2. The opt-in exists because Anthropic official endpoints do not charge these refusals, while a third-party Claude-compatible relay may still charge them. Applying a global exemption blindly could create under-billing.
  3. LMM has additional billing paths (tiered_expr, quota saturation/composition, tool-call surcharges and custom pre-consume settlement), so the exemption must be placed before all final quota composition paths rather than patched only around the upstream call sites.

Proposed LMM-native implementation

Introduce a narrow settlement marker, e.g. ContextKeyBillingExemptReason=claude_refusal_no_output, only when all of these are true:

  • final upstream response is a Claude refusal;
  • no content block/output was produced;
  • upstream usage exists;
  • output_tokens == 0;
  • the selected policy says this channel/provider follows Anthropic no-charge refusal semantics.

Keep the original upstream response/usage returned to the client unchanged. Only the billing/logging view should become zero-cost.

For streaming, track whether any content_block_start has occurred before the refusal. For non-streaming, use len(content) == 0. Cover the Claude-native, Chat Completions -> Claude, and Claude -> Responses paths that share the adaptor.

At settlement time, the exemption should be resolved before ordinary ratio billing, fixed-price billing, tiered_expr, and tool-surcharge composition. It should:

  • settle final quota to 0;
  • fully refund any pre-consumed quota;
  • skip tiered expression settlement for this request;
  • record request count normally but not token/quota consumption;
  • retain the admin-only refusal reason;
  • add a machine-readable public/admin log field such as billing_exempt_reason=claude_refusal_no_output;
  • never mutate the usage body sent to the client.

Policy decision required

Prefer provider/channel capability over a single unconditional global switch if LMM can distinguish Anthropic-billed channels from third-party Claude-compatible relays. If that distinction is not reliable yet, use an explicit default-off setting as upstream #7552 does.

Do not make client identity, group, /fast, price lock, OAuth mode, or admin-AI state influence this decision.

Acceptance criteria

  • Anthropic-style pre-output refusal (content=[], stop_reason=refusal, usage present, output_tokens=0) settles to quota 0 when the no-charge policy is enabled/applicable.
  • Pre-consumed quota is fully refunded.
  • Ratio billing does not charge input tokens for the exempt request.
  • Fixed-price billing does not charge the request.
  • tiered_expr does not reintroduce a non-zero charge.
  • Tool-surcharge composition cannot reintroduce a charge for an otherwise exempt no-output refusal.
  • Mid-stream refusal after any content/output remains normally billable.
  • Empty content with output_tokens>0 remains billable.
  • Refusal without a usage object remains conservatively billable.
  • Non-refusal empty responses are unaffected.
  • Policy disabled / unsupported third-party channel preserves current billing behavior.
  • Native Claude, converted Chat Completions, streaming, non-streaming, and Claude->Responses paths have regression tests.
  • Consume logs distinguish the billing exemption from the existing admin refusal reason.
  • Existing pricing/group/model-price-lock//fast/OAuth/admin-AI behavior is unchanged.

Verification target

At minimum run focused Claude relay tests, service settlement tests (including SQLite full pre-consume -> settle flow), tiered-billing regression tests, go vet on touched packages, and the existing CI billing-safety/release qualification jobs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions