Conversation
Add the llama-deepseek-v41-trace tool under tools/deepseek-v41-trace/: manifest export, prompt builder, Linux containment helper, host attestation, and run/verify scripts for cross-runtime (llama.cpp vs DS4) correctness comparison. Wire the subdirectory into tools/CMakeLists.txt and add the LLAMA_DEEPSEEK_V41_NATIVE_CONTAINMENT_TESTS build option next to the other LLAMA_BUILD_* options. Assisted-by: Claude Sonnet 5 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add server-child-protocol.h with server_child_state_line() to recognize child state lines even when ANSI reset codes precede them (colored logs), and use it in server-models.cpp instead of a raw prefix match. Skip the stop-monitor thread for download children (they never enter stopping_models, so joining it would hang remove()) and notify cv_stop directly on their teardown path. Fix test_router.py download-cache test isolation so tests don't leak state across runs. Add test-server-child-protocol.cpp covering the ANSI-reset and malformed-line cases, wired into CTest. Assisted-by: Claude Sonnet 5 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add test-deepseek41-trace.py, the pytest suite driving the trace harness through model-free manifest, prompt-provenance, containment, and cross-runtime schema-alignment checks. Wire it into CTest as test-deepseek41-trace, feeding it the native trace/manifest/inject binary paths and gating the containment-helper environment variable on LLAMA_DEEPSEEK_V41_NATIVE_CONTAINMENT_TESTS (Linux only). Also register test-deepseek41-runtime.cpp, whose registration PR A deferred pending tools/deepseek-v41-trace/trace-components.h landing with this layer. Assisted-by: Claude Sonnet 5 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
@dzannotti This is layer 2 of the same revival -- full context is on #68. Specific to this PR's original form (#59): the trace-tooling's hardcoded What I'm asking for: review alongside #68, in the same base-to-stacked order the Stack (ggml-org#69) reflects. |
|
Closing and superseding with #71 (stacked on #70, which supersedes #68 for the same reason). Same content, same 3 commits -- but this PR's commits carried a real name + machine hostname as git author/committer ( |
Layer 2 of 2 (PR B): correctness trace harness
AGENT-AUTHORED. Written end-to-end by Claude Sonnet 5 (GitHub Copilot CLI), revived from closed #59.
Stacked on #66 (Layer 1, "deepseek41 : add core DeepSeek V4.1 runtime"): base branch is
jeromecoste-microsoft-deepseek-v41-core-runtime, so this diff is additive on top of #66 and willshrink to just this layer once #66 merges.
Background
#59/#49/#60 were closed together by a reviewer because none were stacked (all targeted
masterdirectly, causing overlapping diffs) and because trace-tooling defaults/README examples leaked a
personal path (
/home/papa/...). This PR re-applies the validated correctness-harness content fromthat effort as fresh, minimal, logical commits (not a replay of the old 113-commit branch), built on
top of #66 instead of
master. The personal-path leak is fixed at the source commit used(
1eeb5bd49d63d6eb3692d92ccd6e161abc3067a9on the old recovery branch) and re-verified here with aclean
git grep -niE papaacross every file this PR touches.What this adds
tools/deepseek-v41-trace/: the cross-runtime correctness trace harness (native trace/prompt-builderbinaries, Linux containment helper, host-attestation, receipt generation, preflight/watchdog-audit
checks, Python trace-format/verification helpers) plus its CMake wiring
(
LLAMA_DEEPSEEK_V41_NATIVE_CONTAINMENT_TESTSoption,tools/CMakeLists.txtsubdirectory).tools/server/server-child-protocol.hand matchingtools/server/server-models.cppfixes: ANSI-resetresidue recognition for child state lines, skip the stop-monitor thread for download children, and
notify
cv_stopdirectly on their teardown -- needed so the trace harness can reliably read childprocess state during the correctness flow.
tools/server/tests/unit/test_router.py: aprivate_router_download_cachefixture for testisolation around the download-cache fixes above.
tests/test-server-child-protocol.cpp(new) andtests/test-deepseek41-trace.py(new, thecross-runtime correctness trace test suite), wired into CTest via
tests/CMakeLists.txt.Nothing from #60's scope (
src/llama-host-memory-guard.*, its loader wiring,test-host-memory-guard.cpp)is included here; that stays out of both layers of this revival.
Scope note: watchdog-revision trust anchor
The harness pins a
WATCHDOG_REVISIONgit-commit-hash as a trust anchor that must be an ancestor of HEADfor
preflight.watchdog_audit()to pass. The value carried over from the old branch pointed at a commitin the old, unrelated fork history, which can never be an ancestor of this repo. Updated it to #66's
actual commit that adds
scripts/strix_memory_watchdog.py(02235de637ae1ffe8aeef9479628962792190b22);script content hash is unchanged (verified byte-identical via
shasum -a 256). If #66 is rebased oramended before merge, this pin will need to be re-verified/updated, or
test_canonical_watchdog_artifacts_embed_and_validate/test_approved_watchdog_is_exactwill fail.Cross-runtime context (no merge dependency)
This harness's manifest/receipt schema is designed to align with the unmerged draft
ggml-org#28696 (safetensors -> GGUF converter for DeepSeek V4.1), so trace artifacts stay
comparable across runtimes as that work lands upstream. It makes no performance claim: DeepSeek V4.1
performance work was, and remains, blocked pending cross-runtime correctness passing through this harness.
Any Fedora/ROCm validation referenced historically in this effort was build/install/trace/containment
evidence only -- explicitly not V4.1 ROCm inference correctness evidence.
An external contributor's independently-benchmarked ROCm DeepSeek V4.1 implementation for the same
hardware class (antirez/ds4#1036, by kyuz0) was reviewed during this revival. It validates this harness's
design assumptions with no correctness gaps surfaced against it: matching Engram row layout, matching
CED 20/20 layer-boundary metadata key
LLM_KV_DSV41_CANDIDATE_SOURCE_LAYER_ID, and matchingattention-sink/split-KV concepts. It is a separate codebase and runtime, not a merge dependency for this
PR, and nothing was ported from it -- it is cited purely as external validation context.
Verified / not verified
Verified (this session, macOS, model-free):
cmake -S . -B build -DLLAMA_BUILD_TESTS=ON -DLLAMA_BUILD_TOOLS=ON -DCMAKE_BUILD_TYPE=Releaseconfiguresand
cmake --build buildcompletes for this layer's targets.ctest -R 'deepseek41|server-child-protocol' --output-on-failure: 8/10 pass.git diff --checkclean; ASCII-only; no personal-path leaks (re-grepped after every checkout).Not verified / known gaps:
by design/policy for this session).
test-deepseek41-traceandtest-deepseek41-trace-install(2 of the 10 above) fail on this Mac with"loaded unclassified module outside trusted system roots: /opt/homebrew/Cellar/openssl@3/.../libcrypto.3.dylib".
Reproduced directly outside pytest -- it is the harness's OpenSSL/Homebrew trusted-roots check hitting a
macOS-specific path layout gap, not a logic bug in this PR. Left as-is per scope; needs a Linux run (or a
macOS-Homebrew-aware trusted-roots entry) to close out, tracked as NOT VERIFIED rather than fixed
blind for one machine.
Assisted-by: Claude Sonnet 5
Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com