Perf/default shared prefix catalog capacity

[giveen]

Problem and scope

normalize_engine_options() defaults the Engine-wide shared-prefix catalog
(max_shared_prefixes) to max(max_concurrency, 4), but a single request can legitimately
produce up to 7 distinct shared-prefix candidates (4 explicit markers + 3 engine-automatic:
tool boundary, leading-instruction boundary, full prompt — src/models/qwen3_5/frontend/frontend.cpp:495,
out.opportunities.reserve(7U)). At low concurrency, the catalog fills from one or two requests'
own candidates. Ordinary client traffic (no explicit cache hints) gets DefaultAutomatic evidence,
which by design can only fill vacant catalog slots and never evict to make room
(docs/maintainer/resource-scheduling-and-context-cache.md, §7.2 evidence table). Once the
undersized catalog fills, affected requests permanently stop getting shared-prefix cache hits and
silently re-prefill their full prompt on every subsequent call — with no error, log warning, or
client-visible signal.

The maintainer's own scheduling doc already stated the correct 7-candidate ceiling two sentences
before citing the wrong constant as the default:
https://github.com/giveen/ninfer/blob/5b4303c0ea0e8ab2be3efa54a677829f3edab6e5/docs/maintainer/resource-scheduling-and-context-cache.md#L508-L514

Scope: this PR changes only the default value of max_shared_prefixes and its documentation. It
does not change the candidate-generation logic, the evidence/pressure model, or any explicitly
configured deployment (--max-shared-prefixes N continues to be honored verbatim).

Implementation

Where the code and docs said 4 (pre-fix, commit 5b4303c):

1. The default computation itself:
   https://github.com/giveen/ninfer/blob/5b4303c0ea0e8ab2be3efa54a677829f3edab6e5/src/runtime/engine/model_instance.cpp#L115-L116
2. The constant it borrowed (meant for explicit-marker count, not total per-request candidates):
   https://github.com/giveen/ninfer/blob/5b4303c0ea0e8ab2be3efa54a677829f3edab6e5/include/ninfer/types.h#L22
3. Public EngineOptions header comment documenting the default as contract:
   https://github.com/giveen/ninfer/blob/5b4303c0ea0e8ab2be3efa54a677829f3edab6e5/include/ninfer/types.h#L130
4. docs/serving.md CLI reference table:
   https://github.com/giveen/ninfer/blob/5b4303c0ea0e8ab2be3efa54a677829f3edab6e5/docs/serving.md#L788
5. The maintainer doc's own default-sizing rationale (self-contradicting the line cited above):
   https://github.com/giveen/ninfer/blob/5b4303c0ea0e8ab2be3efa54a677829f3edab6e5/docs/maintainer/resource-scheduling-and-context-cache.md#L512-L514

Fix, commit 0f5f96f:

• New constant kMaximumPreparedPromptCacheCandidatesPerRequest = 7, named and commented against
  the same 7-candidate ceiling the doc already stated:
  https://github.com/giveen/ninfer/blob/0f5f96f44bee177b83b09b2b15d674501f10291e/include/ninfer/types.h#L22-L24
• Default computation now uses it:
  https://github.com/giveen/ninfer/blob/0f5f96f44bee177b83b09b2b15d674501f10291e/src/runtime/engine/model_instance.cpp#L115-L116
• All three other locations above (header comment, docs/serving.md, maintainer doc rationale)
  updated to max(C,7) / cite the new constant, same commit.
• New regression test for the default computation itself, covering max_concurrency 1/2/8,
  explicit-override preservation, and disabled-cache normalization:
  https://github.com/giveen/ninfer/blob/0f5f96f44bee177b83b09b2b15d674501f10291e/tests/test_context_cache_defaults.cpp

Tradeoffs considered: catalog slot count and actual Device/Host KV residency are independent
capacity axes in this design (docs/maintainer/resource-scheduling-and-context-cache.md, §3.1) —
raising the default from 4 to 7 slots adds bookkeeping/descriptor capacity, not additional KV
memory reservation by itself. I have not directly measured the marginal resident-memory or
workspace cost of the larger catalog; flagging this as unverified rather than asserting it is
negligible.

Verification

Unit-level (ninfer_context_cache_defaults_test, new in this PR): fails against the pre-fix
commit at max_concurrency 1 and 2 (default shared-prefix catalog capacity is smaller than one request's own candidate ceiling); passes post-fix at 1/2/8. Full cmake --build build -j
(478 targets: every product binary, every Op-level GPU test) is clean on both commits. Neighboring
suites unaffected: ninfer_serve_options_test, ninfer_resource_manager_test,
ninfer_kv_capacity_test, ninfer_admission_policy_test, ninfer_context_cost_test,
ninfer_request_log_test all pass unchanged.

End-to-end, real model, workload/hardware/toolchain as described in the linked Issue's
Environment section (Qwen3.8-27B NVFP4, RTX 5090, CUDA 13.3, ninfer-serve --max-concurrency 2 --spec mtp --kv-dtype k8v4 ..., identical flags in both configurations — only
--max-shared-prefixes differed). Methodology: one warmup request discarded per trial, then 5
distinct ~450-token prompts sent via plain POST /v1/chat/completions (no cache hints — ordinary
client traffic), then the same 5 prompts resent and timings.cache_n /
usage.prompt_tokens_details.cached_tokens read from each response. 3 trials per configuration,
fresh server restart before every trial to eliminate cross-trial catalog contamination.

	--max-shared-prefixes 4	default (this PR, = 7)
Round-2 hits, all 3 trials	3/5, 3/5, 3/5	5/5, 5/5, 5/5
Round-2 token-level hit rate	1335/2250 (59.3%), identical all 3 trials	2221/2250 (98.7%), identical all 3 trials
Requests permanently re-prefilling from scratch	2/5	0/5

Zero variance across all 6 trials — the admission decision is deterministic for a fixed catalog
state and prompt sequence, so repetition here confirms reproducibility rather than characterizing
noise.

This is not a universal "N% more cache hits" claim — it is evidence the ceiling is real and the fix
removes it for the specific workload shape that exercises it (several genuinely distinct
shared-prefix candidates at low max_concurrency). Deployments at max_concurrency ≥ 7, or
traffic that never produces more than 4 distinct shared prefixes, were never affected by this bug.

Not verified / limitations:

• Marginal resident-memory or descriptor-bookkeeping cost of a 7-slot vs. 4-slot catalog was not
  directly measured (see Implementation section).
• Did not check for the same undersizing pattern at other call sites of
  kMaximumExplicitPromptCacheMarkers; only the one fixed here was confirmed.
• Did not test at max_concurrency values other than 2 against the real model (unit test covers
  1/2/8 at the config-normalization level only, not end-to-end).