fix(hw-gate): keep Redline capture dumps out of the seat prompts

[Kaden-Schutt]

Summary

First kernel-bucket run (#690, 33892920406): both lanes 4/4 pass, then both seats returned empty text. omp log for the decide launch:

"Pre-prompt context maintenance triggered by pending prompt size", contextTokens: 1357577, contextWindow: 1000000

The Redline report's decode.captures — 516 KB of raw per-dispatch records per lane — was inlined verbatim into the verdict and decide prompts along with the rest of hw-gate.json.

Changes

• run.py: elide_captures() replaces capture-dump lists/dicts over 4 KB (captures/dispatches/packets/raw) with their entry count. Every verdict field (pass, sequence_stable, measurement, aql_shadow, failures) stays; the full report is still on disk as hw-gate-logs/redline.json in the artifact.
• review.py: evidence_for_prompt() caps inlined evidence at 600 KB with an explicit truncation marker — the diff was already capped at 400 KB, the evidence was not.

Evidence

On #690's real evidence: 1,282,896 → 41,435 bytes per lane (hipx 1,002,238 → 39,928); seat prompt ≈ 10 K tokens; pass=True, sequence_stable=True, measurement intact. Backstop on the un-elided report: 614,502 bytes with the marker. python3 -m pytest scripts/hw-gate/tests -q → 103 passed.

Which surface(s) does this touch?

• policy files — scripts/hw-gate/{run,review}.py (hard floor: a human merges this)

[hipfire-sol]

hw-gate sol prelim

summary: This policy change reduces hw-gate seat prompt size in two stages: run.py recursively replaces large list/dict values under capture-like keys with entry-count markers while retaining the complete Redline report on disk, and review.py byte-caps serialized evidence embedded in verdict and decide prompts. It does not change model execution or GPU behavior, but it changes which hardware evidence the reviewing seats can inspect.

run_hardware: true
run_hardware_reasons: The changed Python is readable and contains no encoded payload or unexplained unsafe operation.; run.py continues to invoke the existing local Redline harness and read its local JSON report; the new transformation is in-memory JSON traversal only, and the full report remains at hw-gate-logs/redline.json.; review.py continues its existing git, GitHub CLI, and seat-process operations; the change only serializes and caps evidence passed to those seats.; This is policy code, so hardware execution cannot by itself approve it and a human must review the evidence-loss semantics.

routes:

mode	tag	source	why
—	—	—	no routes

unavailable_routes:

(none)

claim_assessment: The author claims real #690 evidence shrinks from roughly 1.28 MB to 41 KB per lane while preserving pass, sequence_stable, measurement, and related verdict fields, with a 600 KiB fallback cap. Proving that requires the cited real report or equivalent fixture, byte-size assertions, preservation assertions for every decision-bearing field, tests for nested key collisions, and a test showing oversized truncated evidence cannot produce an unsafe approval. The supplied diff adds no tests for either new helper, so the stated 103-test pass establishes only that the prior suite still passes.

questions_for_author:

• Why is elision based solely on four generic key names recursively rather than the documented Redline schema paths?
• Where are the regression tests for elide_captures() and evidence_for_prompt(), including preservation of failure fields and behavior when decisive evidence falls beyond the 600 KiB cap?
• Should an evidence cap produce structured per-section elision or force needs-human instead of presenting arbitrary invalid JSON that may omit a failing fixture?

[hipfire-sol]

hw-gate evidence — 2 lane(s) — verdict pass

lane hiptrx (gfx1201)

hw-gate evidence

field	value
base	931b571c978b976fc6135ab8cd0e92b2c150a5bc
head	eb8dc0aa5871c3ff17bc936d7632ce8377e77f04
buckets
host gfx	gfx1201
host rocm	7.15.26333-0000000
device	3
runner	hiptrx
daemon_md5	17ba7dfab49aa1c1110d3f412f072648
hipfire_md5	1b5fbb40d5df438dad66054d016d119a
build_seconds	39.73681139945984
verdict	pass
logs_dir	hw-gate-logs

fixtures

no fixtures

kernel

not run

lane hipx (gfx1100)

hw-gate evidence

field	value
base	931b571c978b976fc6135ab8cd0e92b2c150a5bc
head	eb8dc0aa5871c3ff17bc936d7632ce8377e77f04
buckets
host gfx	gfx1100
host rocm	7.15.26333-0000000
device	0
runner	hipx
daemon_md5	17ba7dfab49aa1c1110d3f412f072648
hipfire_md5	1b5fbb40d5df438dad66054d016d119a
build_seconds	15.795318841934204
verdict	pass
logs_dir	hw-gate-logs

fixtures

no fixtures

kernel

not run

[hipfire-sol]

hw-gate sol verdict

{
  "claim_verdict": "not-exercised",
  "confidence": 0.96,
  "coverage": {
    "gaps": [
      "No kernel bucket ran, so scripts/hw-gate/run.py:773-807 did not exercise elide_captures() on a Redline report.",
      "The evidence does not include the cited #690 report, before/after byte counts, or assertions that all decision-bearing fields survive elision.",
      "The 600 KiB truncation path in scripts/hw-gate/review.py:653-669 was not exercised.",
      "No regression tests cover generic nested key collisions or failure evidence ordered beyond the prompt cap.",
      "This changes policy code and therefore requires human review regardless of successful hardware execution."
    ],
    "surfaces_evidenced": [
      "under-cap evidence serialization in verdict and decide prompts"
    ],
    "surfaces_touched": [
      "policy",
      "hw-gate evidence serialization",
      "review prompts"
    ]
  },
  "decision": "needs-human",
  "eyeball": [
    "scripts/hw-gate/run.py:786-807: verify that recursively matching generic keys captures, dispatches, packets, and raw cannot remove decision-bearing diagnostics.",
    "scripts/hw-gate/review.py:653-669: assess whether arbitrary byte truncation and invalid trailing JSON may hide later failures or should force a hold/needs-human outcome."
  ],
  "phase": "verdict",
  "rationale": "The hardware run succeeded but selected no buckets or fixtures, so it did not exercise the new Redline elision path or oversized-evidence backstop. The current under-cap evidence reached this seat intact, showing only that the ordinary prompt path works. There is no demonstrated regression warranting block, but the evidence-loss semantics at scripts/hw-gate/run.py:786-807 and scripts/hw-gate/review.py:653-669 remain uncovered, and policy changes require human review.",
  "regressions": []
}

Floor: hard=['policy_paths: scripts/hw-gate/review.py,scripts/hw-gate/run.py'] soft=["coverage_gaps: ['No kernel bucket ran, so scripts/hw-gate/run.py:773-807 did not exercise elide_captures() on a Redline report.', 'The evidence does not include the cited #690 report, before/after byte counts, or assertions that all decision-bearing fields survive elision.', 'The 600 KiB truncation path in scripts/hw-gate/review.py:653-669 was not exercised.', 'No regression tests cover generic nested key collisions or failure evidence ordered beyond the prompt cap.', 'This changes policy code and therefore requires human review regardless of successful hardware execution.']", 'model needs-human'] model_decision=needs-human final=needs-human

[hipfire-sol]

hw-gate sol verdict needs-human: The hardware run succeeded but selected no buckets or fixtures, so it did not exercise the new Redline elision path or oversized-evidence backstop. The current under-cap evidence reached this seat intact, showing only that the ordinary prompt path works. There is no demonstrated regression warranting block, but the evidence-loss semantics at scripts/hw-gate/run.py:786-807 and scripts/hw-gate/review.py:653-669 remain uncovered, and policy changes require human review.

[hipfire-fable]

announcement: Policy files change here, so this holds for a human by rule; the evidence behind it is now real rather than claimed. I replayed the kernel bucket on gfx1201 through your run_kernel() with the manifest fixture: pass, and elide_captures replaced exactly one path (decode.captures), leaving 266 paths byte-identical — 1,122,343 B on disk to a 7,498 B report, per-lane seat evidence 1,254,149 to 12,683 B; on the actual #690 reports the merged evidence goes 3,596,268 to 38,736 B. Two things for the maintainer to weigh: eliding the whole captures list also drops launches/unique_kernels/sequence_hash from the prompt (only stderr_tail still carries them) — eliding redline_capture.sequence instead keeps them; and the 600 KiB backstop hands a seat non-JSON text with no lanes/verdict and no floor reason marking the truncation, so unlike master's fail-closed verdict_parse_failed a seat could greenlight on partial evidence — a truncated flag feeding the soft floor would close that. Neither helper has a test yet; the analysis script under the evidence dir is a usable fixture.

investigation:

| question | route | result | evidence |

|---|---|---|---|

| Does elide_captures() preserve every decision-bearing field on a real Redline report produced by the PR-built daemon, through the PR's actual run_kernel() path? | HIP_VISIBLE_DEVICES=3 CARGO_TARGET_DIR=/home/kaden/actions-runner/_cache/hw-gate-target python3 $HW_GATE_EVIDENCE/kernel-replay-gfx1201/replay_run_kernel.py (imports scripts/hw-gate/run.py from the PR head and calls run.write_isolated_home(HIPFIRE_HOME/kernel-replay,'3',models) then run.run_kernel(checkout, /home/kaden/.hipfire/models, fixtures.json buckets.kernel.redline, manifest, env, EV/hw-gate-logs); underlying harness command: python3 scripts/redline_daemon_harness.py --model /home/kaden/.hipfire/models/qwen3.6-27b.mq4 --daemon $HW_GATE_BIN/daemon --out .../hw-gate-logs/redline.json --pm4 --capture-repeats 2 --measure-repeats 2 --decode-iterations 16) | status=pass exit=0 reason='' in 26.6 s on gfx1201 (daemon md5 17ba7dfab49aa1c1110d3f412f072648, config devices="3"). Raw redline.json 1,122,343 B on disk (640,846 B compact) -> elided report 7,498 B; replaced_paths=["decode.captures"] only, 266 other paths byte-identical, deep_equal_after_removal=true. Retained identical: pass=true, decode.sequence_stable=true, decode.measurement.tok_s median 35.39, prefill.128/512.sequence_stable=true with their captures kept whole (377 B, hash 46e0d1365ba9ea34), aql_shadow bit_exact/blob_bit_exact/gdn_frame_exact=true, dispatches=899, packets=1, aql_contract_probe.kernels=20. Per-lane seat evidence {'kernel': result}: 1,254,149 -> 12,683 B. Dropped from the prompt: decode.captures[i].redline_capture.{launches=899, unique_kernels=20, sequence_hash=a1d4c38f6281130c} and per-capture tok_s; they survive only in stderr_tail: 'decode: stable=True launches=899 kernels=20 hash=a1d4c38f6281130c median=35.4 tok/s'. | redline.json,run_kernel-result.json,elision-compare.json,replay-meta.json,replay.stdout} |

| Do the author's #690 byte counts hold on the actual #690 lane reports, and what does the merged evidence look like with and without elision? | python3 $HW_GATE_EVIDENCE/elision-real-reports/analyze.py (run.elide_captures on ../lanes/hw-gate-evidence-{hipx,hiptrx}/hw-gate-logs/redline.json; merge_evidence.merge() over synthetic lane dirs carrying the raw vs elided kernel report; review.evidence_for_prompt on both) | hipx (gfx1100) 523,274 -> 6,842 B; hiptrx (gfx1201) 640,844 -> 7,501 B; replaced_paths=["decode.captures"] on both; pass, sequence_stable, measurement, prefill captures, aql_shadow.{bit_exact,blob_bit_exact,dispatches=659/899,packets=1}, aql_contract_probe.kernels=16/20 identical raw vs elided; sequence_hash (fe4c3ec486001342 / a1d4c38f6281130c) absent from the elided JSON. Merged evidence (merge_evidence.py:68-69 embeds the first lane's report twice, under kernel and kernels): raw 3,596,268 B (~0.9 M tokens, matches the #690 collapse) -> elided 38,736 B (~9.7 K tokens), untruncated, parses, all 14 top-level keys present including verdict=pass and kernels={hiptrx,hipx}. | {elision-summary.json,elision-hipx.json,elision-hiptrx.json,merged-raw.json,merged-elided.json,prompt-evidence-raw.txt,prompt-evidence-elided.txt,backstop-summary.json} |

| What does the 600 KiB backstop actually hand a seat when elision has not fired, and can a truncated prompt hide a failure from the floor? | review.evidence_for_prompt(merged_raw); review.hard_floor(evidence, select, hw_run_result, []) on merged_raw, merged_elided, a copy with kernel.status='fail', and a copy with a fixtures row attractor=True | Raw prompt = 614,502 B ending with '[hw-gate.json truncated at 614400 bytes of 3596268; ...]'. With sort_keys the cut lands inside 'kernel': only base, binaries, buckets, fixtures, head, host are complete; kernels, lanes, lanes_missing, logs_dir, schema, verdict, version are absent; text does not parse as JSON. hard_floor (review.py:94-150) reads the evidence dict, not the prompt: raw and elided dicts give the same reasons, kernel.status='fail' adds 'kernel status != pass' (line 117), an attractor row adds 'attractor detected' (lines 118-141). So truncation cannot hide a failed kernel/fixture/attractor from the floor, but nothing in the floor records that the seat saw partial evidence. | {prompt-evidence-raw.txt,backstop-summary.json,floor-summary.json} |

| Is eliding by the generic key names {captures, dispatches, packets, raw} safe against the schema the daemon and harness actually emit (Sol's question 1)? | read-only source survey of scripts/redline_daemon_harness.py:443-616, crates/hipfire-engine/src/redline.rs:13-45, crates/hipfire-generate/src/redline.rs (shadow/prefix/dspark/dflash/dispatch-profile builders), crates/hipfire-daemon/src/main.rs:3727-3735,4014-4033; spot-checked by me at crates/hipfire-generate/src/redline.rs:3462-3470 | On the harness path the only >4 KiB list under an elision key is decode.captures (redline_capture.sequence, engine redline.rs:23-45, only emitted with redline_detail). aql_shadow/prefix_shadow/product-route/dspark dispatches and packets are integer counts everywhere; 'raw' is never emitted by any redline response; on bit_exact=false the mismatch detail is booleans (logits_equal/kv_equal/recurrent_equal, redline.rs:3466-3470) plus hash+byte-count snapshots and prefix 'differing' names, never a per-packet/per-dispatch diff list. The one list-shaped 'dispatches' (redline_dispatch_profile, redline.rs:3558-3712) is off the harness path and is boundary metadata beside a separate correctness object. Large keys the set does NOT cover: aql_contract_probe.contracts, pm4_prefix_profile.rows, redline_dispatch_profile.samples[].spans_ns, dflash_verify_shadow.parity.windows — the backstop is what catches those. | redline.rs:3462-3463 ('dispatches': prepared.0, 'packets': prepared.1) |

| Edge cases Sol raised: the run.py:743 'raw' string, failure fields beside an elided key, the 4096 boundary, and the PR's own suite. | python3 $HW_GATE_EVIDENCE/elision-real-reports/analyze.py (edge-case section); python3 -m pytest scripts/hw-gate/tests -q -p no:cacheprovider | (a) {'error':..,'raw':'A'*5000} -> string kept verbatim (run.py:743 parse-failure path is unaffected); (b) failures list kept, a 100-item list under 'raw' replaced; (c) aql_shadow with bit_exact=False, packets list(1000), dispatches=899 -> packets replaced, dispatches int and bit_exact=False kept; (d) captures serializing to exactly 4096 B kept, 4097 B replaced; (e) None and str pass through. pytest: 103 passed in 4.07 s; grep for elide_captures|evidence_for_prompt in scripts/hw-gate/tests/ = 0 hits, so neither helper has a test. | {edge-cases.json,pytest.txt} |

unproven:

• The gfx1100 lane could not be replayed on this host (four gfx1201 devices only); the fix(mq4v2): one MQ-V2 prefill admit rule for llama and qwen35; discriminating GEMM parity; spec §9 #690 hipx report was analysed offline instead — same single replaced path, 523,274 -> 6,842 B.

• A seat actually consuming a truncated prompt end-to-end (the 600 KiB backstop under a live Sol/Fable launch) — no seat can be launched from this sandbox; the prompt text it would receive is on disk at $HW_GATE_EVIDENCE/elision-real-reports/prompt-evidence-raw.txt.

• Harness modes outside the fixture (redline_dispatch_profile, pm4_prefix_profile, dflash/dspark verify shadows) were surveyed at source only, not executed; none is on the kernel bucket's path.

rationale: Hard floor fired on policy_paths (scripts/hw-gate/run.py, scripts/hw-gate/review.py) so this is a hold by construction; the question was whether the evidence-loss semantics Sol flagged are safe, and they are now evidenced. I replayed the kernel bucket on gfx1201 through the PR's own run_kernel() (run.py:696-776, elide at :776) with the manifest fixture (fixtures.json:63-74): pass, and elide_captures (run.py:787-805) replaced exactly one path, decode.captures, leaving 266 paths byte-identical and every verdict field intact ($HW_GATE_EVIDENCE/kernel-replay-gfx1201/elision-compare.json). On the real #690 lane reports the same single path is replaced and the reconstructed merged evidence goes 3,596,268 -> 38,736 B, which is the collapse the PR fixes. The generic-key concern is inert on the current schema: shadow dispatches/packets are counts (crates/hipfire-generate/src/redline.rs:3462-3463), 'raw' is never emitted, and failure detail is booleans plus hash snapshots, never under an elision key. Three things the human should weigh before merging: (1) eliding the whole captures list drops launches/unique_kernels/sequence_hash — the fingerprint a kernel reviewer compares across lanes and against master — from the prompt; they survive only in the 2000-char stderr_tail. Eliding redline_capture.sequence (engine redline.rs:23-45) instead of the parent captures list keeps those ~90 bytes per capture and is schema-precise rather than key-name based. (2) The review.py:659-669 backstop cuts inside 'kernel' under sort_keys, so a seat gets no kernels/lanes/verdict and non-JSON text; the floor is computed from the dict (review.py:94-150, demonstrated in floor-summary.json) so no failed kernel or attractor can be hidden, but nothing records the truncation as a soft floor reason — on master a collapsed seat fails closed via verdict_parse_failed (review.py:196-198, 1157-1158), whereas a seat reading a truncated prompt could greenlight. Returning a truncated flag from evidence_for_prompt and appending e.g. 'evidence_truncated' to soft would restore fail-closed for ~5 lines. (3) Neither helper has a test (0 references in scripts/hw-gate/tests); analyze.py under $HW_GATE_EVIDENCE/elision-real-reports is a ready fixture-driven starting point. One observation outside this diff: the lane artifacts for this run (../lanes/hw-gate-evidence-*/hw-gate-logs/) carry #690's redline.json and battery logs although this run selected no buckets — the reused runner workspace's hw-gate-logs/ leaks stale logs into the artifact, the same class as #705 fixed for fable-evidence.

hard floor: ['policy_paths: scripts/hw-gate/review.py,scripts/hw-gate/run.py']

soft floor: ["coverage_gaps: ['No kernel bucket ran, so scripts/hw-gate/run.py:773-807 did not exercise elide_captures() on a Redline report.', 'The evidence does not include the cited #690 report, before/after byte counts, or assertions that all decision-bearing fields survive elision.', 'The 600 KiB truncation path in scripts/hw-gate/review.py:653-669 was not exercised.', 'No regression tests cover generic nested key collisions or failure evidence ordered beyond the prompt cap.', 'This changes policy code and therefore requires human review regardless of successful hardware execution.']", 'model needs-human']

[hipfire-fable]

Policy files change here, so this holds for a human by rule; the evidence behind it is now real rather than claimed. I replayed the kernel bucket on gfx1201 through your run_kernel() with the manifest fixture: pass, and elide_captures replaced exactly one path (decode.captures), leaving 266 paths byte-identical — 1,122,343 B on disk to a 7,498 B report, per-lane seat evidence 1,254,149 to 12,683 B; on the actual #690 reports the merged evidence goes 3,596,268 to 38,736 B. Two things for the maintainer to weigh: eliding the whole captures list also drops launches/unique_kernels/sequence_hash from the prompt (only stderr_tail still carries them) — eliding redline_capture.sequence instead keeps them; and the 600 KiB backstop hands a seat non-JSON text with no lanes/verdict and no floor reason marking the truncation, so unlike master's fail-closed verdict_parse_failed a seat could greenlight on partial evidence — a truncated flag feeding the soft floor would close that. Neither helper has a test yet; the analysis script under the evidence dir is a usable fixture. Hard floor fired on policy_paths (scripts/hw-gate/run.py, scripts/hw-gate/review.py) so this is a hold by construction; the question was whether the evidence-loss semantics Sol flagged are safe, and they are now evidenced. I replayed the kernel bucket on gfx1201 through the PR's own run_kernel() (run.py:696-776, elide at :776) with the manifest fixture (fixtures.json:63-74): pass, and elide_captures (run.py:787-805) replaced exactly one path, decode.captures, leaving 266 paths byte-identical and every verdict field intact ($HW_GATE_EVIDENCE/kernel-replay-gfx1201/elision-compare.json). On the real #690 lane reports the same single path is replaced and the reconstructed merged evidence goes 3,596,268 -> 38,736 B, which is the collapse the PR fixes. The generic-key concern is inert on the current schema: shadow dispatches/packets are counts (crates/hipfire-generate/src/redline.rs:3462-3463), 'raw' is never emitted, and failure detail is booleans plus hash snapshots, never under an elision key. Three things the human should weigh before merging: (1) eliding the whole captures list drops launches/unique_kernels/sequence_hash — the fingerprint a kernel reviewer compares across lanes and against master — from the prompt; they survive only in the 2000-char stderr_tail. Eliding redline_capture.sequence (engine redline.rs:23-45) instead of the parent captures list keeps those ~90 bytes per capture and is schema-precise rather than key-name based. (2) The review.py:659-669 backstop cuts inside 'kernel' under sort_keys, so a seat gets no kernels/lanes/verdict and non-JSON text; the floor is computed from the dict (review.py:94-150, demonstrated in floor-summary.json) so no failed kernel or attractor can be hidden, but nothing records the truncation as a soft floor reason — on master a collapsed seat fails closed via verdict_parse_failed (review.py:196-198, 1157-1158), whereas a seat reading a truncated prompt could greenlight. Returning a truncated flag from evidence_for_prompt and appending e.g. 'evidence_truncated' to soft would restore fail-closed for ~5 lines. (3) Neither helper has a test (0 references in scripts/hw-gate/tests); analyze.py under $HW_GATE_EVIDENCE/elision-real-reports is a ready fixture-driven starting point. One observation outside this diff: the lane artifacts for this run (../lanes/hw-gate-evidence-*/hw-gate-logs/) carry #690's redline.json and battery logs although this run selected no buckets — the reused runner workspace's hw-gate-logs/ leaks stale logs into the artifact, the same class as #705 fixed for fable-evidence.