No valid checkpoint to restore + Max length reached on multi-turn tool calls (Qwen3.6-MoE, NPU)

[winaes13-ux]

Summary

On multi-turn tool-call conversations, flm serve fails with No valid checkpoint to restore followed by Max length reached, stopping prefilling.... The model never produces a final answer — the second turn (after the tool result is returned) is rejected.

Environment

FLM version	1.0.5
Model	qwen3.6-moe:35b-a3b (NPU2)
NPU firmware	1.1.2.64
Platform	Linux, AMD Ryzen AI 9 HX 470 (Strix Halo)
Client	OpenCode 1.18.31 (OpenAI-compatible API)
Endpoint	POST /v1/chat/completions

Reproduction

1. Start flm serve qwen3.6-moe:35b-a3b --ctx-len 262144 --pmode performance.
2. Send a request with tools defined. The model correctly emits a <tool_call> and finish_reason = tool_calls.
3. Send a follow-up request with the same conversation plus the assistant (tool_calls) and tool (result) messages.
4. FLM fails — see log below.

Log

[FLM]  Use cached prompt!
[FLM]  Matched 2 out of 4 messages (2 new to prefill).
[FLM]  Start prefill...
[FLM]  Total images: 0
[FLM]  No valid checkpoint to restore
[WARNING]  Max length reached, stopping prefilling...
[FLM]  Creating checkpoint at context length 0
[🔵 ]  NPU Lock Released!

The server returns:

{"error":{"message":"Max length reached!","type":"model_error","code":400}}

Root cause

Qwen3_6_MOE::insert() calls qwen3_6_moe_npu::restore() when meta_info.restore_allowed == true. On the second turn, restore() returns a negative value (no valid checkpoint), but the method then unconditionally assigns:

this->total_tokens = restore_idx;   // becomes -1
this->token_history = checkpoint_his;

checkpoint_his still holds the previous turn's token history (~16k tokens). total_tokens is now desynchronized from the actual KV-cache size. _shared_insert() then checks:

if (this->total_tokens + tokens.size() >= this->MAX_L) {
    header_print("WARNING", "Max length reached, stopping prefilling...");
    return false;
}

Because total_tokens was never reset on restore failure, the check fires even though the actual context is well below MAX_L (32 768). clear_context() is never called, so the KV cache is left in an inconsistent state.

Proposed fix

In common/AutoModel/modeling_qwen3_6_moe.cpp, inside Qwen3_6_MOE::insert():

1. Handle restore() failure explicitly — reset state instead of assigning a negative restore_idx:

     if (meta_info.restore_allowed) {
         restore_idx = qwen3_6_moe_engine->restore();
-        this->total_tokens = restore_idx;
-        this->token_history = checkpoint_his;
+        if (restore_idx < 0) {
+            this->lm_engine->clear_context();
+            this->total_tokens = 0;
+            this->token_history.clear();
+            this->checkpoint_his.clear();
+        } else {
+            this->total_tokens = restore_idx;
+            this->token_history = checkpoint_his;
+        }
     }

2. Re-sync total_tokens with token_history after _shared_insert() — guarantees the counter reflects the real KV-cache size regardless of what happened in restore() / clear_context() / _shared_insert():

     bool success = has_images
         ? this->_shared_insert(meta_info, tokens, is_cancelled, &image_payload, last_image_token_index)
         : this->_shared_insert(meta_info, tokens, is_cancelled, nullptr);
 
+    if (this->total_tokens != (int)this->token_history.size()) {
+        this->total_tokens = this->token_history.size();
+    }
+
     checkpoint_his = token_history;
     int checkpoint_idx = qwen3_6_moe_engine->checkpoint();
     return success;
 }

Verification

After applying the patch and rebuilding FLM, the same multi-turn tool call now completes successfully. Full log:

[FLM]  Use cached prompt!
[FLM]  Matched 2 out of 4 messages (2 new to prefill).
[FLM]  Start prefill...
[FLM]  Total images: 0
[FLM]  No valid checkpoint to restore
[FLM]  Prefill chunk 1/5 with 4096 tokens
[FLM]  Prefill chunk 2/5 with 4096 tokens
[FLM]  Prefill chunk 3/5 with 4096 tokens
[FLM]  Prefill chunk 4/5 with 4096 tokens
[FLM]  Prefill chunk 5/5 with 303 tokens
[FLM]  Creating checkpoint at context length 16687
[FLM]  Start generating...
[FLM]  Model RAW Output:
Вот список проиндексированных проектов:
... (correct answer)

The model now returns the expected final answer instead of a 400 error.

Impact

Tool-calling agents (OpenCode, Claude Code, Cline, and any OpenAI-compatible client that performs multi-step tool calls) are unusable with flm serve on qwen3.6-moe without this fix. The bug affects all multi-turn tool interactions, not just a specific client.

---

[winaes13-ux]

Addendum: default_context_length sizing for qwen3.6-moe

Context

The patch described above fixes the restore() failure path, but it does not remove the need for a correctly sized default_context_length. Without a sufficiently large MAX_L, the same multi-turn session still fails later with Max length reached once the prompt has grown enough.

Why default_context_length matters

When restore() returns a negative value, the patched Qwen3_6_MOE::insert() resets total_tokens to 0 and clears token_history / checkpoint_his. _shared_insert() then performs a full prefill of the entire prompt:

total_tokens (0) + tokens.size() (full prompt) >= MAX_L ?

If MAX_L is smaller than the prompt size, the prefill is rejected:

[FLM]  No valid checkpoint to restore
[WARNING]  Max length reached, stopping prefilling...
[FLM]  Creating checkpoint at context length 0

and the server returns HTTP 400:

{"error":{"message":"Max length reached!","type":"model_error","code":400}}

MAX_L is read from model_list.json → default_context_length. For qwen3.6-moe:35b-a3b the upstream value is 32 768, which is not enough for long agent sessions.

Sizing rule

A full re-prefill happens on top of whatever was already in the KV cache before the reset, so MAX_L must cover the worst-case prompt with a 2× headroom:

worst_case_prompt = system_prompt
                  + tools_definition
                  + conversation_history
                  + max_tokens

default_context_length = 2 * worst_case_prompt

Component sizes (OpenCode, OpenAI-compatible client)

Component	Typical size	Notes
system_prompt	~2 000 tokens	OpenCode system prompt + AGENTS.md + MCP hints
tools_definition	~6 000 tokens	20 MCP tools: codebase-memory-mcp (15) + context7 (2) + built-ins
conversation_history	~16 000 tokens	typical multi-turn agent session
max_tokens	4 096 tokens	request-level max_tokens
worst_case_prompt	~28 096 tokens	sum of the above
default_context_length	65 536	2 × 28 096 = 56 192 → rounded up

Rounding table

worst_case_prompt	2 × worst_case_prompt	default_context_length (rounded)
≤ 16 384	≤ 32 768	32 768
≤ 32 768	≤ 65 536	65 536 ← current OpenCode configuration
≤ 65 536	≤ 131 072	131 072
≤ 131 072	≤ 262 144	262 144 (native Qwen3.6-MoE maximum)
> 131 072	> 262 144	262 144 (clamped to model maximum)

Workaround used locally

Until the model-side fix lands, the user-side copy of model_list.json is patched via FLM_CONFIG_PATH:

export FLM_CONFIG_PATH="$HOME/.config/flm/model_list.json"
export FLM_XCLBIN_PATH="/opt/fastflowlm/share/flm"

mkdir -p "$(dirname "$FLM_CONFIG_PATH")"
cp /opt/fastflowlm/share/flm/model_list.json "$FLM_CONFIG_PATH"

# raise default_context_length for qwen3.6-moe to 65536
sed -i 's/"default_context_length": 32768/"default_context_length": 65536/g' \
    "$FLM_CONFIG_PATH"

FLM_CONFIG_PATH is read by common/utils.cpp (get_models_directory() / get_model_path()), and the user copy survives FLM upgrades, unlike the system copy under /opt/fastflowlm/share/flm/.

Verification

With the patch and default_context_length = 65 536 in place, a 28-message agent session completes without Max length reached. The longest observed prefill in the test session was:

[FLM]  Prefill chunk 1/8 with 4096 tokens
...
[FLM]  Prefill chunk 8/8 with 2178 tokens
[FLM]  Creating checkpoint at context length 30850
[FLM]  Start generating...

30 850 < 32 768 on the first attempt, but the failure path adds another full prefill on top of it — which is why MAX_L must be sized for 2 × worst_case_prompt, not 1 × worst_case_prompt.

Suggested fix in the repository

1. Apply the insert() patch from the main report.
2. Either raise default_context_length for qwen3.6-moe:35b-a3b in model_list.json to a value ≥ 2 × worst_case_prompt for the intended client configuration, or document the sizing rule so users can adjust it themselves.
3. Optionally, expose default_context_length as a runtime override (e.g. --ctx-len) that is independent of the model registry, so users don’t need to edit model_list.json at all.

---