RFC / Not for merge: Experimental fixes for agent workloads (+10% to dflash2 agent)

docs/maintainer/resource-scheduling-and-context-cache.md

@@ -510,8 +510,8 @@ candidates：全部 tools 之后、连续 leading System/Developer 之后，以
 因此每个请求最多七个 prepared candidates。这个固定上限不是启动配置。
 
 Shared catalog 是 Engine-wide 公共容量，不是每条 lineage 的配额。启用 context cache 时，默认 logical
-capacity 同时覆盖 active concurrency 下限和单请求最多四个显式 markers，即
-`max(max_concurrency, kMaximumExplicitPromptCacheMarkers)`；显式配置仍完整覆盖默认值。这个下限允许较早的
+capacity 同时覆盖 active concurrency 下限和单请求最多七个 prepared candidates，即
+`max(max_concurrency, kMaximumPreparedPromptCacheCandidatesPerRequest)`；显式配置仍完整覆盖默认值。这个下限允许较早的
 稳定层与较晚的滚动 marker 同时成为 owner，但是否 capture、保留或替换仍只由通用 portfolio/pressure
 planning 决定，不提供 Claude、compact 或 token-position 特例。
 

docs/serving.md

@@ -111,7 +111,7 @@ The endpoint supports:
 - `temperature`, `top_p`, presence/frequency penalties, and signed integer `seed`;
 - the compatible `top_k` (`0..20`) and `min_p` (`0..1`) sampler extensions;
 - up to four non-empty stop strings, applied to both reasoning and answer output;
-- `n:1`, text-only `modalities`, and `response_format: {"type":"text"}`;
+- `n:1`, text-only `modalities`, and `response_format` (`{"type":"text"}`, `{"type":"json_object"}`, or `{"type":"json_schema"}`);
 - non-streaming responses and server-sent event streams;
 - `stream_options.include_usage`;
 - llama.cpp-compatible terminal `timings`, plus opt-in `timings_per_token` and
@@ -125,7 +125,7 @@ The endpoint supports:
 - Assistant `reasoning_content` and `reasoning` history aliases.
 
 Options whose observable behavior the Engine cannot provide are rejected when they request that
-behavior. This includes JSON constrained output, nonzero `logit_bias`, requested log probabilities,
+behavior. This includes nonzero `logit_bias`, requested log probabilities,
 audio/file input or audio output, `strict:true`, required or named tool choice,
 `parallel_tool_calls:false` with enabled tools, explicit low/high image detail, web search,
 moderation, low/high verbosity, stored Chat Completions, and non-empty legacy `functions`.
@@ -787,7 +787,7 @@ The table lists executable defaults. The startup example selects a long-context
 | `--host-state-slots N` | pinned Host StateImage capacity | `8` |
 | `--host-kv-mib N` | shared pinned Host Main/Backend KV byte capacity in MiB | `8192` |
 | `--max-private-continuations N` | private continuation descriptor capacity | `2 * max-concurrency` |
-| `--max-shared-prefixes N` | Engine-wide shared stable-prefix descriptor capacity | `max(max-concurrency, 4)` |
+| `--max-shared-prefixes N` | Engine-wide shared stable-prefix descriptor capacity | `max(max-concurrency, 7)` |
 | `--max-long-anchors-per-continuation N` | private long-anchor limit per continuation | `2` |
 | `--no-thinking` | disable thinking by default | thinking on |
 | `--preserve-thinking` | preserve closed-turn assistant reasoning by default | off |

include/ninfer/types.h

@@ -19,7 +19,8 @@ using TokenId = std::int32_t;
 
 inline constexpr std::uint32_t kMaximumConcurrency               = 8;
 inline constexpr std::size_t kMaximumContextCacheSessionKeyBytes = 256;
-inline constexpr std::size_t kMaximumExplicitPromptCacheMarkers  = 4;
+inline constexpr std::size_t kMaximumExplicitPromptCacheMarkers              = 4;
+inline constexpr std::size_t kMaximumPreparedPromptCacheCandidatesPerRequest = 7;
 // Aggregate encoded image/video payload retained by one prompt, independent of item count.
 inline constexpr std::size_t kMaximumPromptMediaBytes    = 256ULL << 20;
 inline constexpr std::size_t kDefaultMediaCacheBytes     = 1ULL << 30;
@@ -127,7 +128,7 @@ struct StartupObserver {
 
 struct ContextCacheOptions {
     // Engine resolves every optional once at construction. With C=max_concurrency, the enabled
-    // defaults are H=C, R=8, Host KV=8 GiB, P=2C, S=max(C,4) and L=2;
+    // defaults are H=C, R=8, Host KV=8 GiB, P=2C, S=max(C,7) and L=2;
     // Engine::options() returns those effective values.
     bool enabled = true;
     // Extra Device checkpoint StateImage slots H. Total Device StateImage capacity is C + H.

src/models/qwen3_5/frontend/chat_template.cpp

@@ -160,11 +160,8 @@ RenderedChat CompiledChatTemplate::render(const std::vector<ChatMessage>& messag
     Json context = template_parameters(options, special_tokens_);
     if (continuation) {
         const auto& final = messages.back();
-        if (final.role != ChatRole::Assistant || final.has_media() ||
-            !final.reasoning_content.empty() || !final.tool_calls.empty() ||
-            context.value("enable_thinking", false)) {
-            throw std::invalid_argument("assistant continuation requires a final text-only "
-                                        "assistant message and disabled thinking");
+        if (final.role != ChatRole::Assistant || final.has_media()) {
+            throw std::invalid_argument("assistant continuation requires a final assistant message without media");
         }
     }
     context["continue_final_message"] = continuation;