Skip to content

Anthropic cache_control markers produce no shared-prefix reuse on the released v0.4.0 serve #29

Description

@alphastorm

Measured 2026-08-31 on the released 5090 container (binary 7eb66643, one shared-prefix slot configured). Five /v1/messages requests sharing an identical ~83.5K-token system block with cache_control: {"type": "ephemeral"} on the system text block all classified reuse=root, prefix_cache_hit_tokens=0 - seed and every fan cost ~34 s each. The schema parser does record system cache boundaries (anthropic_schema.cpp flatten_system_value), so either the boundary never becomes a SharedStablePrefix capture group, shared capture never runs at completion, or the follow-up shortlist never matches.

Receipt context: omp-ninfer docs/measurements/2026-08-31-fanout-probe-main.json (the Responses-path fanout in the same session: branch-from-older-id is always root as well). Together these mean no same-lane fanout path reuses a shared base on the released build; the v0.4.x engine train (upstream 099d9032 value-aware shared scheduling, 59febed prefix retention, plus Responses-path markers) is the fix vehicle. Diagnosis starting point: trace a marked anthropic request through frontend capture-group construction to the shared catalog and the next request's shortlist.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions