diff --git a/BACKLOG.md b/BACKLOG.md index 7c3d029a..54b18ee7 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -31,7 +31,7 @@ CC-001/CC-002 were consumed by PR #24 fix bundle inline, with no standalone entr | CC-509 | ✅ closed 2026-07-22 | detached gate launch liveness：對 sandbox parent-death 早期死亡 fail-loud，提供 supervisor readiness／identity evidence | arch/gate | 2026-07-22 | pr:#440 | P2 | hygiene | | CC-510 | ✅ closed 2026-07-23 | Codex detached dispatch continuation：App Server callback、authenticated completion envelope 與 foreground fallback | arch/DX | 2026-07-23 | pr:#443 | P2 | design | | CC-511 | ⚠️ partial 2026-07-24 | ship publish authorization：Phase A current-tree authoritative full-suite 已交付；Phase B review-closure evidence 仍待 CC-515／CC-517 | release/gate | 2026-07-23 | pr:#446 | P1 | design | -| CC-512 | ⚠️ partial 2026-07-27 | Slice A 已交付 coordinate sources／CLI resolution；machine envelope、evidence capture 與 shared verifier 仍待 Slice B／C | ops/gate | 2026-07-23 | — | P1 | design | +| CC-512 | ✅ closed 2026-07-27 | Slices A／B／C 已交付：coordinate sources／CLI resolution、machine-owned assurance envelope／evidence capture、shared verifier／parity ratchets；targeted 不再是 tier | ops/gate | 2026-07-23 | pr:#451 | P1 | design | | CC-513 | 🔵 active | canonical gate policy resolver：minimum tier、required reviewers、mode recommendation 與 downgrade audit | security/gate | 2026-07-23 | — | P1 | design | | CC-514 | 🔵 active | orthogonal delivery assurance map、machine-derived tables 與 feature/docs/high-risk recipes | docs/process | 2026-07-23 | — | P2 | design | | CC-515 | 🔵 active | gate artifact immutable subject、freshness 與 consumer applicability shared verifier | arch/gate | 2026-07-23 | — | P1 | design | @@ -41,6 +41,7 @@ CC-001/CC-002 were consumed by PR #24 fix bundle inline, with no standalone entr | CC-519 | 🔵 active | selected-reviewer coverage／finding contract：declared coverage、stable IDs 與 actionable fix boundary | ops/gate | 2026-07-23 | — | P1 | design | | CC-520 | 🔵 active | synthesis parity 與 remediation seed：findings union、root-cause grouping、coverage matrix 與 no-silent-drop | ops/gate | 2026-07-23 | — | P1 | design | | CC-521 | 🔵 active | test-gap matrix、protocol recovery 與 live recall evaluation 分層 | ops/test | 2026-07-23 | — | P2 | design | +| CC-522 | 🔵 active | 任意 `--test-cmd` 的 opaque／structured capability negotiation、執行失敗分類與外部 evidence recovery | ops/test | 2026-07-27 | feedback:2026-07-27 | P1 | design | | CC-465 | 🔵 active | memory/context 關鍵詞管線 CJK 支援：抽出共用零依賴斷詞 lib，取代三處各自 ASCII-only 抽詞；工作序列起點（465→467→468→466）（2026-07-07 記憶系統深入分析） | memory | 2026-07-07 | feedback:2026-07-07 | P2 | retrieval | | CC-466 | ⏸ deferred | 記憶卡片生命週期閉環：expires_at 執行 + 關窗式 supersede + usage sidecar 休眠偵測 + doctor→distill 接線；僅在 CC-467 證明 stale/dormant card 已形成實際問題時啟動 | memory | 2026-07-07 | feedback:2026-07-07 | P2 | retrieval | | CC-467 | 🔵 active | `pmctl memory stats`：注入效益可視化（唯讀聚合器）——注入 bytes/卡片命中分佈/從未命中卡/episode 填寫率，回答「記憶有跟沒有差在哪」；排在 CC-466 之前（2026-07-07；業界僅離線 recall 評測，無 per-injection 遙測） | DX/memory | 2026-07-07 | — | P2 | retrieval | @@ -1616,7 +1617,7 @@ authorization。 --- -## CC-512 — tier／mode／pass／coverage／independence assurance 正交化 ⚠️ partial 2026-07-27 +## CC-512 — tier／mode／pass／coverage／independence assurance 正交化 ✅ 2026-07-27 **Problem**: runtime 雖已將 tier detection、reviewer selection 與 `SEQUENTIAL=true|false` 分開，但目前仍有三個 truth gap： @@ -1695,11 +1696,12 @@ authorization。 invalid/conflicting inputs，以及 dispatch brief 中的 requested/resolved／coverage coordinates。`--tier full --reviewers critic` 保留 full intent + critic-only selection；新 producer／verifier 尚未交付。 -2. **B — machine-owned envelope + evidence capture（pending）**：sequential combined session、 - parallel per-reviewer/synthesis sessions、targeted initial reference、copy-mode - truthful degradation。 -3. **C — verifier + remaining parity ratchets（pending）**：claim consistency、v1 legacy - classification、result/help/docs parity、affected-test mapping。 +2. **B — machine-owned envelope + evidence capture（✅ delivered 2026-07-27, + pr:#451）**：sequential combined session、parallel per-reviewer/synthesis sessions、 + targeted initial reference、copy-mode truthful degradation。 +3. **C — verifier + remaining parity ratchets（✅ delivered 2026-07-27, + pr:#451）**：claim consistency、v1 legacy classification、result/help/docs parity、 + affected-test mapping。 **Done-when**: @@ -1722,6 +1724,17 @@ engine、FSM或 mandatory parallel policy。 [[CC-513]]再產 policy resolution，[[CC-515]]再把 structural evidence 與 subject/freshness/applicability接起來。 +**Outcome**: Slices A／B／C 已完成。Gate producer 現在以分離的 portable policy +sources解析 tier、mode、pass、coverage與independence，並產生 machine-owned +assurance sidecar；dispatch outcome、canonical run evidence、result／repository／ +subject bindings與protected attestation皆由 runtime 擷取，不再由 reviewer Markdown +自述。Shared verifier會檢查 structural／claim consistency、coverage partition、 +mode／topology、targeted initial reference與v1 legacy降級；copy-mode fallback由 +shared verifier產生並受parity ratchet保護。Formal full gate為GO且assurance verified， +current-tree authoritative full suite為97 passed、0 failed、0 skipped。 + +**See**: pr:#451 + **Cross-link**: [[CC-513]]、[[CC-515]]、[[CC-518]]、[[CC-519]]、 `docs/review-model.md`。 @@ -2119,6 +2132,92 @@ deterministic fail closed，後者具模型波動，不應混成 CI hard gate。 --- +## CC-522 — `--test-cmd` execution outcome 與 evidence capability 分層 🔵 active + +**Framing**: 本票強化既有 `pmctl gate run --test-cmd` pre-flight 與 qa-tester +對測試執行證據的解讀，不重寫 gate 流程。任意可執行 shell command 永遠是合法輸入； +structured result 是 opt-in capability，不是導入 gate 前必須先改造各 repo runner 的 +門檻。本票保留 [[CC-491]] 的 portable opaque evidence 與 structured reusable +evidence 分工；tier／mode／pass／coverage／independence 仍由 [[CC-512]] 擁有， +subject freshness／consumer applicability 仍由 [[CC-515]] 擁有，test-gap內容仍由 +[[CC-521]] 擁有。禁止新增 gate kind、workflow engine、強制 runner migration， +或以 stdout/stderr 關鍵字猜測 assertion／環境失敗。 + +**Problem**: `--test-cmd` 可能是任意 legacy command，未必產生建議的 structured +result；即使 command 有執行，也可能因 reviewer sandbox、依賴、網路、資源限制或 +timeout 非零退出，而同一 tree 在外部環境可正常通過。目前 pre-flight 雖能保存 +opaque evidence 並在內部辨識 timeout／stale／invalid，最後仍把所有非 PASS 合併成 +一般 test FAIL／NO-GO；qa-tester 也把 non-runnable／flaky 一律視為 block。這會把 +「沒有可用 authorization evidence」誤寫成「diff 已證明有 defect」，同時迫使使用者 +為了避免 false block 先投入 runner 格式改造。 + +**Requirement**: + +1. capability negotiation 必須是漸進式： + - command 未寫 structured sink 時，接受 portable opaque evidence； + - command 寫出 schema-valid result 時，提升為 structured evidence； + - command 有寫 sink 但內容 malformed／subject 不符時標 `invalid-evidence`， + 不得靜默降級 opaque。 +2. machine outcome 分開記錄 command execution、test verdict、evidence richness 與 + authorization applicability。closed execution classification 至少涵蓋 + `pass|test-fail|timeout|environment-error|stale|invalid-evidence| + unclassified-nonzero`；opaque 非零不得靠 log heuristic 自動宣稱 `test-fail`。 +3. 只有 subject-valid structured assertion/test failure 可產生機械 test NO-GO。 + `timeout|environment-error|unclassified-nonzero` 使 operation + `INCOMPLETE/non-authorizing`，保留 command digest、exit、timeout、log digest、 + tree fingerprint與 recovery instructions，但不得冒充 diff-caused reviewer + blocker。Opaque PASS 只證明該 command 對該 subject exit 0，不宣稱 suite coverage + 完整或可作 no-duplicate reuse。 +4. qa-tester output 增加 `inconclusive` run result、failure class 與 evidence refs。 + 已有 outer pre-flight PASS 時不得反射性重跑 full suite，只能追加 scope-bounded + targeted checks；reviewer sandbox 的 timeout／environment failure回報 + inconclusive，只有可歸因 assertion failure、diff-caused coverage gap或測試 + anti-pattern 可 block。 +5. qa-tester 在執行任何可能耗時的自主測試前，必須先寫入並 flush early + checkpoint，至少含已完成 matrix/audit、預定 command、開始時間、timeout budget、 + evidence refs 與 `run.status: running`。測試必須經 bounded shell-owned wrapper + 執行，持續保存 stdout/stderr log、process exit／timeout 與最後可觀察進度； + reviewer session被外層 watchdog終止時，gate仍機械產生 + `partial/inconclusive` artifact，列出完成／未完成 sections、checkpoint與 log + pointer。不得只依賴模型在 command 返回後才首次寫檔，也不得讓 timeout留下 + 0-byte／無結果。 +6. sequential combined session與parallel reviewer session都必須保留上述 qa + checkpoint/result；partial qa artifact不是有效 reviewer verdict，synthesis不得 + 將它補寫成 pass／block或納入正常 findings union，operation只能 + `INCOMPLETE/non-authorizing`。若模型在 checkpoint 前違規直接執行長測試，wrapper + 仍須留下 shell-owned attempt/log evidence並明示 `checkpoint: missing`。 +7. 外部執行 evidence recovery 必須驗證同一 repository subject、HEAD/tree + fingerprint、command digest、suite identity與 artifact integrity；符合 + [[CC-515]] freshness/applicability 才能取代 inconclusive。口頭／純 log PASS + 可作 manual clue，不得單獨授權 GO。不得自動重跑或提高 timeout 掩蓋 performance + regression；重跑由使用者明示或 policy-bounded recovery 觸發並記錄 attempts。 +8. human result 明確區分 `code/test NO-GO`、`gate INCOMPLETE` 與 + `evidence unavailable`，提供可複製的 same-command／adjusted-timeout／external + evidence recovery 指令，不要求使用者先採用 structured producer。 +9. deterministic fixtures 覆蓋：opaque PASS、opaque nonzero、structured PASS／ + test-fail、sink missing、sink malformed、timeout、environment error、tree drift、 + external evidence subject match/mismatch、qa targeted failure、timeout 前已寫 + checkpoint、checkpoint 前違規執行仍有 shell log、sequential／parallel partial + preservation，以及不得把 inconclusive轉成 blocker或 GO。 + +**Done-when**: 任意 legacy `--test-cmd` 不需格式改造即可得到 truthful opaque +evidence；structured producer可獲得更強 reuse/coverage 語意；環境／timeout失敗會 +fail closed 但不誤報產品 defect；qa-tester與 gate artifact對同一 execution class +給出一致、可恢復的結論；qa自主測試即使 timeout 也必有非空 checkpoint、attempt +metadata與 log pointer。 + +**Non-goals**: 不保證任意 command 可自動判斷失敗根因；不解析自由文字 log 作 +authorization；不降低 current-tree test evidence要求；不讓 external PASS 省略 +subject/digest驗證；不在本票建立通用 CI provider integration。 + +**Dependencies**: outcome/capability Phase A 複用 [[CC-470]]／[[CC-491]] 可先行； +external reusable evidence Phase B 依賴 [[CC-515]]。與 [[CC-521]] 的 test-gap／ +protocol recovery contract保持正交。P1。 + +**Cross-link**: [[CC-470]]、[[CC-491]]、[[CC-512]]、[[CC-515]]、[[CC-521]]。 + +--- + ## CC-508 — 所有間接 dispatch 的 parent-operation control plane ✅ 2026-07-25 **Problem**: `pmctl gate run`、`pmctl ship --parallel`／adapter 路徑、`pmctl task dispatch` 與任何未來 producer 都可能以一個 parent operation 間接啟動一或多個 detached dispatch；但產品控制面主要只暴露個別 `pmctl dispatch cancel `。parent ID 與其子 run 沒有強制、可查的 ownership relation，也沒有一致的 producer-level cancel surface。當任一 producer 卡住、選錯 executor 或需中止時，操作者無法透過 pmctl 取消整個 operation；直接對 supervisor PID 操作會繞過 run state、sentinel 與 cancel-vs-complete 單一終態契約，並可能留下無法判定的 stale operation。 diff --git a/MILESTONES.md b/MILESTONES.md index 77897732..8ca85d75 100644 --- a/MILESTONES.md +++ b/MILESTONES.md @@ -107,7 +107,7 @@ | 票 | 摘要 | 狀態 | |----|------|------| -| CC-512 | Slice A coordinate sources／CLI resolution 已交付；machine-owned assurance envelope、evidence capture、verifier 仍待 B／C；targeted 不再是 tier | ⚠️ | +| CC-512 | Slices A／B／C：coordinate sources／CLI resolution、machine-owned assurance envelope／evidence capture、shared verifier／parity ratchets；targeted 不再是 tier | ✅ pr:#451 | | CC-513 | canonical resolver：minimum tier、required reviewers、recommended/required mode、generic vs maintainer policy 與 downgrade audit | 🔵 | | CC-515 | immutable subject；artifact validity、subject freshness、policy applicability 三軸 shared verifier | 🔵 | @@ -122,6 +122,7 @@ | CC-519 | selected-reviewer coverage/finding contract；sequential logical sections 與 parallel session isolation 分開 | 🔵 | | CC-520 | synthesis findings-union parity、root-cause grouping、coverage matrix、remediation seed、no silent drop | 🔵 | | CC-521 | actionable test-gap matrix + bounded protocol recovery；seeded live recall 僅作 quality evaluation | 🔵 | +| CC-522 | arbitrary `--test-cmd` opaque／structured negotiation；test failure 與 timeout／environment INCOMPLETE 分流 | 🔵 | ### Phase 9 — maintainer closure + publish authorization diff --git a/commands/pr-gate.md b/commands/pr-gate.md index 494b93ec..0709652b 100644 --- a/commands/pr-gate.md +++ b/commands/pr-gate.md @@ -277,8 +277,19 @@ When the `pmctl gate wait` background Bash completion notification arrives: brief failure summary: exit code + last ~20 lines of the supervisor log at `pmctl artifacts show --cd ""`). 4. Read `result_file` directly (both executor routes write it in-process). To - re-confirm out of band, run `pmctl gate verify ` (the - literal path parsed in step 2, not a shell variable; exit 0 = valid). + re-confirm out of band, run `pmctl gate verify ` from the + repository the gate reviewed (use the literal path parsed in step 2, not a + shell variable; exit 0 = valid). + New results must report `assurance: verified` and point to a sibling + `gate_assurance_v2` JSON file. A legacy result or unbound v1 envelope may report + `assurance: unavailable`; do not treat that as proof of tier/mode/coverage + or reviewer-session independence. For repo-layout results that claim + verified independence, verification also requires the protected producer + attestation, a result under that repository's canonical state partition, + and matching canonical terminal run records. The producer publishes the + sidecar before the v2 result that references it; verification briefly + retries when it observes an in-flight v2 result before its protected + attestation rename completes. 5. Prepend `PR-gate complete.` to completion relay and include the full gate result (including `Final: GO` / `Final: NO-GO`) unchanged. 6. On failure, avoid collapsing findings; relay the actual stderr summary and diff --git a/core/README.md b/core/README.md index 3426c7d4..558479fb 100644 --- a/core/README.md +++ b/core/README.md @@ -2,9 +2,8 @@ This directory contains the canonical PM-runtime data contract. **`core/` is definitions only — it knows nothing executable.** -- `schema/` — JSON Schema files (`.schema.json`) for the 8 first-class - entities: Task, Run, Event, Review, Decision, Brief, - Handover, ContextPack. +- `schema/` — JSON Schema files (`.schema.json`) for runtime entities + and evidence envelopes, including gate assurance. - `policy/` — declarative YAML/TSV tables for enums, presets, and state machines. - `state/` — definition of the on-disk state-store layout @@ -38,4 +37,7 @@ The designated writer module in `runtime/lib/state-writer.sh` is the sole manage ## Schema versioning -Every payload schema includes `schema_version: { const: 1 }` as a required field. Future breaking changes bump the int; old payloads remain valid against the old schema version. `jq '.schema_version'` is the bash-readable discriminator. No `$id` URLs. +Every payload schema includes `schema_version` as a required integer `const` +field. Breaking changes bump that integer; old payloads remain valid against +the schema version that defines them. `jq '.schema_version'` is the +bash-readable discriminator. No `$id` URLs. diff --git a/core/schema/gate-assurance.schema.json b/core/schema/gate-assurance.schema.json new file mode 100644 index 00000000..18ba364d --- /dev/null +++ b/core/schema/gate-assurance.schema.json @@ -0,0 +1,155 @@ +{ + "title": "Gate assurance envelope", + "description": "Machine-owned portable gate assurance envelope.", + "type": "object", + "required": ["kind", "schema_version", "result", "bindings", "coordinates", "dispatch", "provenance"], + "properties": { + "kind": { "const": "gate_assurance_v2" }, + "schema_version": { "const": 2 }, + "result": { + "type": "object", + "required": ["final"], + "properties": { "final": { "enum": ["GO", "NO-GO"] } }, + "additionalProperties": false + }, + "bindings": { + "type": "object", + "required": [ + "result_sha256", + "repo_root", + "repo_identity", + "base_commit", + "head_commit", + "subject_fingerprint" + ], + "properties": { + "result_sha256": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "repo_root": { "type": "string", "pattern": "^/" }, + "repo_identity": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "base_commit": { "type": "string", "pattern": "^[a-f0-9]{40}$" }, + "head_commit": { "type": "string", "pattern": "^[a-f0-9]{40}$" }, + "subject_fingerprint": { "type": "string", "pattern": "^[a-f0-9]{64}$" } + }, + "additionalProperties": false + }, + "coordinates": { + "type": "object", + "required": ["tier", "mode", "pass", "coverage", "independence"], + "properties": { + "tier": { + "type": "object", + "required": ["requested", "resolved", "evidence_floor"], + "properties": { + "requested": { "enum": ["auto", "express", "standard", "full"] }, + "resolved": { "enum": ["express", "standard", "full"] }, + "evidence_floor": { "type": "string", "minLength": 1 } + }, + "additionalProperties": false + }, + "mode": { + "type": "object", + "required": ["requested", "resolved", "topology", "synthesis"], + "properties": { + "requested": { "enum": ["default", "sequential", "parallel"] }, + "resolved": { "enum": ["sequential", "parallel"] }, + "topology": { "enum": ["combined-session", "per-reviewer-sessions"] }, + "synthesis": { "enum": ["inline", "separate-session"] } + }, + "additionalProperties": false + }, + "pass": { + "type": "object", + "required": ["requested", "resolved", "scope", "initial_result"], + "properties": { + "requested": { "enum": ["initial", "targeted"] }, + "resolved": { "enum": ["initial", "targeted"] }, + "scope": { "enum": ["comprehensive", "remediation-delta"] }, + "initial_result": { "type": ["string", "null"] } + }, + "additionalProperties": false + }, + "coverage": { + "type": "object", + "required": ["requested", "selected", "skipped", "vocabulary"], + "properties": { + "requested": { + "oneOf": [ + { "type": "null" }, + { "$ref": "#/definitions/reviewerSet" } + ] + }, + "selected": { "$ref": "#/definitions/reviewerSet" }, + "skipped": { "$ref": "#/definitions/reviewerSet" }, + "vocabulary": { "$ref": "#/definitions/reviewerSet" } + }, + "additionalProperties": false + }, + "independence": { + "type": "object", + "required": [ + "implementation_context_isolated", + "reviewer_topology", + "per_reviewer_independent", + "evidence_status" + ], + "properties": { + "implementation_context_isolated": { "type": ["boolean", "null"] }, + "reviewer_topology": { "enum": ["combined-session", "per-reviewer-sessions"] }, + "per_reviewer_independent": { "type": ["boolean", "null"] }, + "evidence_status": { "enum": ["verified", "unavailable", "unverified"] } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "dispatch": { + "type": "object", + "required": ["outcomes"], + "properties": { + "outcomes": { + "type": "array", + "items": { + "type": "object", + "required": ["role", "reviewer", "status", "run_id", "evidence_status"], + "properties": { + "role": { "enum": ["combined", "reviewer", "synthesis", "preflight"] }, + "reviewer": { "type": ["string", "null"] }, + "status": { "enum": ["passed", "failed", "skipped"] }, + "run_id": { "type": ["string", "null"] }, + "evidence_status": { "enum": ["verified", "unavailable", "unverified"] } + }, + "additionalProperties": false + } + } + }, + "additionalProperties": false + }, + "provenance": { + "type": "object", + "required": ["producer", "policy_source", "attestation"], + "properties": { + "producer": { "const": "pr-gate.sh" }, + "policy_source": { "enum": ["canonical", "generated-snapshot", "mixed"] }, + "attestation": { + "oneOf": [ + { "type": "null" }, + { "type": "string", "pattern": "^gate-assurance-[0-9]{8}-[0-9]{6}\\.attestation\\.json$" } + ] + } + }, + "additionalProperties": false + } + }, + "definitions": { + "reviewerSet": { + "type": "array", + "uniqueItems": true, + "items": { + "type": "string", + "pattern": "^[a-z0-9][a-z0-9-]*$" + } + } + }, + "additionalProperties": false +} diff --git a/docs/review-model.md b/docs/review-model.md index 8d4f883f..45eb309c 100644 --- a/docs/review-model.md +++ b/docs/review-model.md @@ -171,6 +171,25 @@ The portable policy sources are [`core/policy/gate-modes.tsv`](../core/policy/gate-modes.tsv), and [`core/policy/gate-pass-kinds.tsv`](../core/policy/gate-pass-kinds.tsv). +The final producer writes `pr_gate_result_v2` Markdown plus a sibling +`gate_assurance_v2` JSON envelope. The Markdown contains human findings and a +bounded relative `gate_assurance` pointer; the shell-owned envelope records +requested/resolved coordinates, selected/skipped coverage, actual dispatch +outcomes, run IDs, subject commits/fingerprint, and the evidence status behind +independence claims. Repo-layout results with verified independence also carry +a shell-owned attestation in the protected gate run directory. `pmctl gate +verify` validates result/sidecar digests and resolves every claimed run ID +against the canonical terminal records for the invoking repository and rejects +self-consistent artifacts outside that repository's resolved state partition. +The producer publishes the sidecar before atomically replacing the +self-contained v1 result with the v2 result that references it, so interruption +cannot strand a v2 result with a missing sidecar. The protected attestation is +published afterward; verification uses a bounded retry when it observes that +in-flight canonical v2 finalization. Legacy +`pr_gate_result_v1` and unbound `gate_assurance_v1` artifacts remain +structurally readable, but verification reports `assurance: unavailable`; +consumers must not infer mode, coverage, or independence from them. + --- ## When line-by-line review is appropriate diff --git a/runtime/bin/pr-gate.sh b/runtime/bin/pr-gate.sh index 4cabee9c..56989d33 100755 --- a/runtime/bin/pr-gate.sh +++ b/runtime/bin/pr-gate.sh @@ -453,51 +453,292 @@ if [[ -r "$GATE_RESULT_VERIFY_PATH" ]]; then . "$GATE_RESULT_VERIFY_PATH" else # Inline fallback for copy-mode (pr-gate.sh run standalone without runtime/lib/). - # MUST stay in sync with runtime/lib/gate-result-verify.sh; the copy-mode - # regression test exercises this path. - gate_result_verify() { + # Generated from runtime/lib/gate-result-verify.sh by + # scripts/sync-gate-result-verifier-fallback.sh. Do not edit this block by hand. + # gate-result-verifier-fallback:start + gate_result_verdict_verify() { local result_file=${1-} expected_final=${2-} route_label=${3-gate} local final_count frontmatter_final body_final [[ $# -ge 1 && $# -le 3 ]] || { - printf 'gate-result-verify: gate_result_verify expects [expected_final] [route_label]\n' >&2 + printf 'gate-result-verify: gate_result_verdict_verify expects [expected_final] [route_label]\n' >&2 return 2 } - if [[ ! -s "$result_file" ]]; then printf 'Error: %s did not produce the result file: %s\n' "$route_label" "$result_file" >&2 printf 'Gate aborted -- the executor session may have exited 0 without writing a verdict.\n' >&2 return 1 fi - final_count=$(grep -cE '^Final: (GO|NO-GO)$' "$result_file" || true) if [[ "$final_count" -ne 1 ]]; then printf 'Error: gate result file must contain exactly one Final: GO/NO-GO line (found %d): %s\n' \ "$final_count" "$result_file" >&2 return 1 fi - frontmatter_final=$(awk 'BEGIN{s=0} /^---$/ { if (s == 0) { s=1; next } else if (s == 1) { exit } } s && $1 == "final:" { print $2; exit }' "$result_file") if [[ -z "$frontmatter_final" ]]; then printf 'Error: gate result YAML frontmatter missing required field: final: (%s)\n' "$result_file" >&2 return 1 fi - body_final=$(grep -E '^Final: (GO|NO-GO)$' "$result_file" | awk '{print $2}') if [[ "$frontmatter_final" != "$body_final" ]]; then printf 'Error: frontmatter final: (%s) does not match body Final: (%s) in gate result: %s\n' \ "$frontmatter_final" "$body_final" "$result_file" >&2 return 1 fi - if [[ -n "$expected_final" && "$body_final" != "$expected_final" ]]; then printf 'Error: %s verdict (%s) contradicts shell-computed verdict (%s) -- gate result may have been manipulated: %s\n' \ "$route_label" "$body_final" "$expected_final" "$result_file" >&2 return 1 fi + } - return 0 + _gate_result_frontmatter_value() { + local result_file="$1" key="$2" + awk -v wanted="$key" ' + BEGIN{s=0} + /^---$/ { if (s == 0) { s=1; next } else if (s == 1) { exit } } + s && $1 == wanted ":" { print $2; exit } + ' "$result_file" } + + _gate_result_sha256_file() { + local file="$1" digest="" + if command -v sha256sum >/dev/null 2>&1 \ + && digest="$(sha256sum -- "$file" 2>/dev/null | awk '{print $1}')" \ + && [[ -n "$digest" ]]; then + printf '%s\n' "$digest" + return 0 + fi + if command -v shasum >/dev/null 2>&1 \ + && digest="$(shasum -a 256 -- "$file" 2>/dev/null | awk '{print $1}')" \ + && [[ -n "$digest" ]]; then + printf '%s\n' "$digest" + return 0 + fi + printf 'Error: no sha256sum or shasum found -- cannot verify gate assurance binding\n' >&2 + return 2 + } + + gate_assurance_verify() { + local result_file="$1" assurance_file="$2" body_final="$3" + local markdown_tier markdown_mode result_sha assurance_kind + command -v jq >/dev/null 2>&1 || { + printf 'Error: gate assurance verification requires jq\n' >&2 + return 2 + } + if [[ ! -s "$assurance_file" ]]; then + printf 'Error: gate assurance sidecar missing or empty: %s\n' "$assurance_file" >&2 + return 1 + fi + markdown_tier="$(_gate_result_frontmatter_value "$result_file" tier)" + markdown_mode="$(_gate_result_frontmatter_value "$result_file" mode)" + assurance_kind="$(jq -r '.kind // empty' "$assurance_file" 2>/dev/null)" + if [[ "$assurance_kind" == gate_assurance_v1 ]]; then + jq -e --arg final "$body_final" --arg markdown_tier "$markdown_tier" \ + --arg markdown_mode "$markdown_mode" ' + .kind == "gate_assurance_v1" and .schema_version == 1 and + .result.final == $final and + .coordinates.tier.resolved == $markdown_tier and + .coordinates.mode.resolved == $markdown_mode + ' "$assurance_file" >/dev/null || { + printf 'Error: legacy gate assurance sidecar failed claim verification: %s\n' \ + "$assurance_file" >&2 + return 1 + } + GATE_ASSURANCE_BOUND=false + export GATE_ASSURANCE_BOUND + return 0 + fi + result_sha="$(_gate_result_sha256_file "$result_file")" || return $? + jq -e --arg final "$body_final" --arg result_sha "$result_sha" \ + --arg markdown_tier "$markdown_tier" \ + --arg markdown_mode "$markdown_mode" ' + def only_keys($allowed): + type == "object" and ((keys_unsorted - $allowed) | length) == 0; + def strings_unique: + type == "array" and all(.[]; type == "string" and length > 0) and + (length == (unique | length)); + def same_set($a; $b): ($a | sort) == ($b | sort); + only_keys(["kind","schema_version","result","bindings","coordinates", + "dispatch","provenance"]) and + (.result | only_keys(["final"])) and + (.bindings | only_keys(["result_sha256","repo_root","repo_identity", + "base_commit","head_commit","subject_fingerprint"])) and + (.coordinates | only_keys(["tier","mode","pass","coverage","independence"])) and + (.coordinates.tier | only_keys(["requested","resolved","evidence_floor"])) and + (.coordinates.mode | only_keys(["requested","resolved","topology","synthesis"])) and + (.coordinates.pass | only_keys(["requested","resolved","scope","initial_result"])) and + (.coordinates.coverage | + only_keys(["requested","selected","skipped","vocabulary"])) and + (.coordinates.independence | + only_keys(["implementation_context_isolated","reviewer_topology", + "per_reviewer_independent","evidence_status"])) and + (.dispatch | only_keys(["outcomes"])) and + (all(.dispatch.outcomes[]; + only_keys(["role","reviewer","status","run_id","evidence_status"]))) and + (.provenance | only_keys(["producer","policy_source","attestation"])) and + .kind == "gate_assurance_v2" and .schema_version == 2 and + .result.final == $final and + .bindings.result_sha256 == $result_sha and + (.bindings.repo_root | type == "string" and startswith("/")) and + (.bindings.repo_identity | test("^[a-f0-9]{64}$")) and + (.bindings.base_commit | test("^[a-f0-9]{40}$")) and + (.bindings.head_commit | test("^[a-f0-9]{40}$")) and + (.bindings.subject_fingerprint | test("^[a-f0-9]{64}$")) and + .coordinates.tier.resolved == $markdown_tier and + .coordinates.mode.resolved == $markdown_mode and + .provenance.producer == "pr-gate.sh" and + (.provenance.policy_source | + IN("canonical","generated-snapshot","mixed")) and + (.coordinates.tier.evidence_floor | type == "string" and length > 0) and + (.coordinates.tier.requested == "auto" or + (.coordinates.tier.requested == .coordinates.tier.resolved and + (.coordinates.tier.requested | IN("express","standard","full")))) and + (.coordinates.tier.resolved | IN("express","standard","full")) and + (.coordinates.mode.requested | IN("default","sequential","parallel")) and + (.coordinates.mode.resolved | IN("sequential","parallel")) and + (.coordinates.mode.requested == "default" or + .coordinates.mode.requested == .coordinates.mode.resolved) and + ((.coordinates.mode.resolved == "sequential" and + .coordinates.mode.topology == "combined-session" and + .coordinates.mode.synthesis == "inline") or + (.coordinates.mode.resolved == "parallel" and + .coordinates.mode.topology == "per-reviewer-sessions" and + .coordinates.mode.synthesis == "separate-session")) and + (.coordinates.pass.requested | IN("initial","targeted")) and + (.coordinates.pass.resolved | IN("initial","targeted")) and + .coordinates.pass.requested == .coordinates.pass.resolved and + ((.coordinates.pass.resolved == "initial" and + .coordinates.pass.scope == "comprehensive" and + .coordinates.pass.initial_result == null) or + (.coordinates.pass.resolved == "targeted" and + .coordinates.pass.scope == "remediation-delta" and + (.coordinates.pass.initial_result | type == "string" and length > 0))) and + (.coordinates.coverage.vocabulary | strings_unique) and + (.coordinates.coverage.selected | strings_unique) and + (.coordinates.coverage.skipped | strings_unique) and + ((.coordinates.coverage.selected + .coordinates.coverage.skipped) | strings_unique) and + same_set(.coordinates.coverage.selected + .coordinates.coverage.skipped; + .coordinates.coverage.vocabulary) and + (.coordinates.coverage.requested == null or + ((.coordinates.coverage.requested | strings_unique) and + same_set(.coordinates.coverage.requested; + .coordinates.coverage.selected))) and + (.dispatch.outcomes | type == "array") and + (all(.dispatch.outcomes[]; + (.role | IN("combined","reviewer","synthesis","preflight")) and + (if .role == "reviewer" + then (.reviewer | type == "string" and length > 0) + else .reviewer == null + end) and + (.status | IN("passed","failed","skipped")) and + (.evidence_status | IN("verified","unavailable","unverified")) and + (.run_id == null or + (.run_id | type == "string" and test("^run-[A-Za-z0-9]+-[A-Za-z0-9]+$"))))) and + (if .coordinates.mode.resolved == "parallel" and + ([.dispatch.outcomes[] | select(.role == "preflight")] | length) == 0 + then + (.dispatch.outcomes | length) == + ((.coordinates.coverage.selected | length) + 1) and + (all(.dispatch.outcomes[]; + (.role == "reviewer" or .role == "synthesis"))) and + ([.dispatch.outcomes[] | select(.role == "reviewer") | .reviewer] | sort) == + (.coordinates.coverage.selected | sort) and + ([.dispatch.outcomes[] | select(.role == "synthesis")] | length) == 1 + elif .coordinates.mode.resolved == "sequential" and + ([.dispatch.outcomes[] | select(.role == "preflight")] | length) == 0 + then + (.dispatch.outcomes | length) == 1 and + .dispatch.outcomes[0].role == "combined" + else + (.dispatch.outcomes | length) == 1 and + .dispatch.outcomes[0].role == "preflight" and + .dispatch.outcomes[0].status == "failed" + end) and + (.coordinates.independence.evidence_status | + IN("verified","unavailable","unverified")) and + (if .coordinates.independence.evidence_status == "verified" + then (.provenance.attestation | + type == "string" and + test("^gate-assurance-[0-9]{8}-[0-9]{6}\\.attestation\\.json$")) + else .provenance.attestation == null + end) and + .coordinates.independence.reviewer_topology == + .coordinates.mode.topology and + (if .coordinates.independence.evidence_status == "verified" + then + .coordinates.independence.implementation_context_isolated == true and + (all(.dispatch.outcomes[]; .evidence_status == "verified" and .run_id != null)) and + ([.dispatch.outcomes[].run_id] | length == (unique | length)) and + (if .coordinates.mode.resolved == "parallel" + then .coordinates.independence.per_reviewer_independent == true + else .coordinates.independence.per_reviewer_independent == false + end) + else + .coordinates.independence.implementation_context_isolated != true and + .coordinates.independence.per_reviewer_independent != true and + (all(.dispatch.outcomes[]; .evidence_status != "verified")) + end) + ' "$assurance_file" >/dev/null || { + printf 'Error: gate assurance sidecar failed structural/claim verification: %s\n' \ + "$assurance_file" >&2 + return 1 + } + GATE_ASSURANCE_BOUND=true + export GATE_ASSURANCE_BOUND + } + + gate_result_verify() { + local result_file=${1-} expected_final=${2-} route_label=${3-gate} + local version pointer result_parent assurance_file body_final + [[ $# -ge 1 && $# -le 3 ]] || { + printf 'gate-result-verify: gate_result_verify expects [expected_final] [route_label]\n' >&2 + return 2 + } + gate_result_verdict_verify "$result_file" "$expected_final" "$route_label" || return $? + version="$(_gate_result_frontmatter_value "$result_file" gate_result_version)" + case "$version" in + pr_gate_result_v1) + GATE_RESULT_ASSURANCE=unavailable + unset GATE_RESULT_ASSURANCE_FILE + export GATE_RESULT_ASSURANCE + return 0 + ;; + pr_gate_result_v2) + pointer="$(_gate_result_frontmatter_value "$result_file" gate_assurance)" + if [[ -z "$pointer" || "$pointer" == */* || "$pointer" == "." || "$pointer" == ".." \ + || ! "$pointer" =~ ^[A-Za-z0-9._-]+\.json$ ]]; then + printf 'Error: pr_gate_result_v2 requires a bounded sibling gate_assurance pointer: %s\n' \ + "$result_file" >&2 + return 1 + fi + result_parent="$(cd "$(dirname "$result_file")" && pwd -P)" || return 1 + assurance_file="$result_parent/$pointer" + body_final=$(grep -E '^Final: (GO|NO-GO)$' "$result_file" | awk '{print $2}') + gate_assurance_verify "$result_file" "$assurance_file" "$body_final" || return $? + if [[ "${GATE_ASSURANCE_BOUND:-false}" == true ]]; then + GATE_RESULT_ASSURANCE=verified + else + GATE_RESULT_ASSURANCE=unavailable + fi + GATE_RESULT_ASSURANCE_FILE="$assurance_file" + export GATE_RESULT_ASSURANCE GATE_RESULT_ASSURANCE_FILE + ;; + *) + printf 'Error: unsupported or missing gate_result_version in gate result: %s\n' \ + "$result_file" >&2 + return 1 + ;; + esac + } + + # gate-result-verifier-fallback:end +fi + +if ! command -v jq >/dev/null 2>&1; then + printf 'Error: pr-gate requires jq on PATH to produce and verify gate assurance\n' >&2 + exit 2 fi # ── Resolve assurance policy coordinates ───────────────────────────────────── @@ -955,6 +1196,44 @@ pmctl_gate_dispatch_lib_load() { return 0 } +_gate_dispatch_capture() { + local brief_file="$1" run_id="$2" status="$3" + local brief_base role=combined reviewer="" capture_file capture_tmp r + [[ -n "${GATE_ASSURANCE_CAPTURE_DIR:-}" ]] || return 0 + brief_base="$(basename "$brief_file")" + if [[ "$brief_base" == *-synthesis.md ]]; then + role=synthesis + else + for r in ${REVIEWERS:-}; do + if [[ "$brief_base" == *-"$r".md ]]; then + role=reviewer + reviewer="$r" + break + fi + done + fi + capture_file="$GATE_ASSURANCE_CAPTURE_DIR/${role}${reviewer:+-$reviewer}.json" + capture_tmp="$(mktemp "$GATE_ASSURANCE_CAPTURE_DIR/.capture.XXXXXX")" || { + printf 'Error: unable to create private gate dispatch capture\n' >&2 + return 1 + } + if ! jq -n --arg role "$role" --arg reviewer "$reviewer" --arg status "$status" \ + --arg run_id "$run_id" \ + '{role:$role,reviewer:(if $reviewer == "" then null else $reviewer end), + status:$status,run_id:$run_id,evidence_status:"verified"}' > "$capture_tmp"; then + rm -f -- "$capture_tmp" + return 1 + fi + _gate_assurance_destination_check "$capture_file" || { + rm -f -- "$capture_tmp" + return 1 + } + mv -- "$capture_tmp" "$capture_file" || { + rm -f -- "$capture_tmp" + return 1 + } +} + # Gate reviewers are producer children, not opaque adapter processes. Route # each invocation through pmctl's detached dispatch lifecycle, then wait for # its authenticated terminal sentinel. The optional parent id is injected by @@ -1009,11 +1288,19 @@ pmctl_gate_dispatch_and_wait() { printf 'pr-gate: dispatch returned invalid run id\n' >&2 return 2 fi + local dispatch_status=passed + rc=0 ( pmctl_gate_dispatch_lib_load || exit 2 pmctl_dispatch_wait "$PMCTL_DISPATCH_ROOT" "$run_id" --cd "$working_dir" --timeout "$timeout" - ) - rc=$? + ) || { + rc=$? + dispatch_status=failed + } + _gate_dispatch_capture "$brief_file" "$run_id" "$dispatch_status" || { + rm -f "$dispatch_brief" + return 1 + } rm -f "$dispatch_brief" return "$rc" } @@ -1308,9 +1595,11 @@ NUM_REVIEWERS=$(printf '%s\n' "$REVIEWERS" | awk '{print NF}') # Compute skipped dimensions SKIPPED="" +SKIPPED_WORDS="" for r in $ALL_REVIEWERS; do if ! printf '%s' "$REVIEWERS" | grep -qw "$r"; then SKIPPED="${SKIPPED:+$SKIPPED, }$r" + SKIPPED_WORDS="${SKIPPED_WORDS:+$SKIPPED_WORDS }$r" fi done SKIPPED_DISPLAY="${SKIPPED:-none}" @@ -1363,6 +1652,11 @@ TIMESTAMP=$(date +%Y%m%d-%H%M%S) _ARTIFACT_ROOT="${GATE_RUN_DIR_OVERRIDE:-$WORK_DIR}" BRIEF_DIR="$_ARTIFACT_ROOT/.gate-briefs" mkdir -p "$BRIEF_DIR" +GATE_ASSURANCE_CAPTURE_DIR="$(mktemp -d "/tmp/pm-gate-assurance-${TIMESTAMP}.XXXXXX")" || { + printf 'Error: unable to create private gate assurance capture directory\n' >&2 + exit 1 +} +command -p chmod 700 "$GATE_ASSURANCE_CAPTURE_DIR" || exit 1 # Route executor traces (adapter JSONL/last/stderr) to the run dir when provided. # PM_DISPATCH_TRACE_DIR is read by dispatch_via (lib and copy-mode) to forward # --trace-dir to the adapter, so the adapter's own trace files follow the run dir. @@ -1386,6 +1680,50 @@ mkdir -p "$(dirname "$OUTPUT_FILE")" _output_parent="$(cd "$(dirname "$OUTPUT_FILE")" && pwd -P)" OUTPUT_FILE="$_output_parent/$(basename "$OUTPUT_FILE")" unset _output_parent +ASSURANCE_FILE="${OUTPUT_FILE}.assurance.json" +ASSURANCE_POINTER="$(basename "$ASSURANCE_FILE")" +ASSURANCE_ATTESTATION_FILE="" +ASSURANCE_ATTESTATION_POINTER="" +GATE_ASSURANCE_RUNS_FILE="" +if [[ -n "$GATE_RUN_DIR_OVERRIDE" && -n "$PMCTL_DISPATCH_LIB_DIR" ]]; then + ASSURANCE_ATTESTATION_POINTER="gate-assurance-${TIMESTAMP}.attestation.json" + ASSURANCE_ATTESTATION_FILE="$GATE_RUN_DIR_OVERRIDE/$ASSURANCE_ATTESTATION_POINTER" + GATE_ASSURANCE_RUNS_FILE="$( + # shellcheck source=runtime/lib/state-paths.sh + . "$PMCTL_DISPATCH_LIB_DIR/state-paths.sh" + cd "$WORK_DIR" || exit 1 + _SW_REPO_ROOT="$WORK_DIR" _sw_project_dir + )runs.jsonl" +fi + +_gate_assurance_destination_check() { + local path="$1" nlink + if [[ -L "$path" ]]; then + printf 'Error: gate assurance destination must not be a symlink: %s\n' "$path" >&2 + return 1 + fi + if [[ -e "$path" ]]; then + if [[ ! -f "$path" ]]; then + printf 'Error: gate assurance destination must be a regular file: %s\n' "$path" >&2 + return 1 + fi + if nlink="$(stat -c '%h' "$path" 2>/dev/null)"; then + : + elif nlink="$(stat -f '%l' "$path" 2>/dev/null)"; then + : + else + printf 'Error: unable to inspect gate assurance destination link count: %s\n' \ + "$path" >&2 + return 1 + fi + if [[ ! "$nlink" =~ ^[0-9]+$ || "$nlink" -ne 1 ]]; then + printf 'Error: gate assurance destination must not be hardlinked: %s\n' \ + "$path" >&2 + return 1 + fi + fi +} + if [[ -n "$INITIAL_RESULT_RESOLVED" \ && ( "$OUTPUT_FILE" == "$INITIAL_RESULT_RESOLVED" \ || ( -e "$OUTPUT_FILE" && "$OUTPUT_FILE" -ef "$INITIAL_RESULT_RESOLVED" ) ) ]]; then @@ -1393,6 +1731,14 @@ if [[ -n "$INITIAL_RESULT_RESOLVED" \ "$INITIAL_RESULT_RESOLVED" >&2 exit 2 fi +if [[ -n "$INITIAL_RESULT_RESOLVED" \ + && ( "$ASSURANCE_FILE" == "$INITIAL_RESULT_RESOLVED" \ + || ( -e "$ASSURANCE_FILE" && "$ASSURANCE_FILE" -ef "$INITIAL_RESULT_RESOLVED" ) ) ]]; then + printf 'Error: the assurance sidecar must not overwrite the referenced --initial-result: %s\n' \ + "$INITIAL_RESULT_RESOLVED" >&2 + exit 2 +fi +_gate_assurance_destination_check "$ASSURANCE_FILE" || exit 2 touch "$OUTPUT_FILE" # Track all brief files for EXIT cleanup @@ -1408,6 +1754,7 @@ cleanup_briefs() { rm -rf -- "$REVIEWER_DEFINITION_DIR" rmdir "$WORK_DIR/.gate-briefs" 2>/dev/null || true fi + rm -rf -- "${GATE_ASSURANCE_CAPTURE_DIR:-}" } # Relocate gate result artifacts out of the repo when a run dir was supplied. @@ -1450,6 +1797,221 @@ gate_exit_cleanup() { } trap gate_exit_cleanup EXIT +gate_finalize_assurance() { + local result_file="$1" assurance_file="$2" + local final requested_json outcomes_json independence_status implementation_isolated + local per_reviewer_independent expected_count capture_count assurance_tmp result_tmp + local result_sha assurance_sha attestation_tmp run_ids_json + local -a capture_files=() + + final="$(grep -E '^Final: (GO|NO-GO)$' "$result_file" | awk '{print $2}')" + [[ -n "$final" ]] || { + printf 'Error: cannot finalize gate assurance without a unique final verdict\n' >&2 + return 1 + } + if [[ -n "$REVIEWERS_OVERRIDE" ]]; then + requested_json="$(jq -nc --arg reviewers "$REVIEWERS" \ + '$reviewers | split(" ") | map(select(length > 0))')" + else + requested_json=null + fi + + while IFS= read -r _capture_file; do + capture_files+=("$_capture_file") + done < <(find "$GATE_ASSURANCE_CAPTURE_DIR" -maxdepth 1 -type f -name '*.json' -print | LC_ALL=C sort) + capture_count="${#capture_files[@]}" + + if [[ "$PREFLIGHT_STATUS" == fail ]]; then + outcomes_json='[{"role":"preflight","reviewer":null,"status":"failed","run_id":null,"evidence_status":"unavailable"}]' + independence_status=unavailable + implementation_isolated=null + per_reviewer_independent=null + elif [[ -n "$PMCTL_DISPATCH_LIB_DIR" \ + && -n "$ASSURANCE_ATTESTATION_FILE" \ + && -n "$GATE_ASSURANCE_RUNS_FILE" ]]; then + if [[ "$SEQUENTIAL" == true ]]; then expected_count=1; else expected_count=$((NUM_REVIEWERS + 1)); fi + if [[ "$capture_count" -ne "$expected_count" ]]; then + printf 'Error: gate dispatch evidence incomplete (expected %d capture(s), found %d)\n' \ + "$expected_count" "$capture_count" >&2 + return 1 + fi + outcomes_json="$(jq -s '.' "${capture_files[@]}")" || return 1 + independence_status=verified + implementation_isolated=true + if [[ "$SEQUENTIAL" == true ]]; then + per_reviewer_independent=false + else + per_reviewer_independent=true + fi + else + independence_status=unavailable + implementation_isolated=null + per_reviewer_independent=null + if [[ "$SEQUENTIAL" == true ]]; then + outcomes_json='[{"role":"combined","reviewer":null,"status":"passed","run_id":null,"evidence_status":"unavailable"}]' + else + outcomes_json="$(jq -nc --arg reviewers "$REVIEWERS" ' + ($reviewers | split(" ") | map(select(length > 0))) as $selected | + ([$selected[] | {role:"reviewer",reviewer:.,status:"passed",run_id:null, + evidence_status:"unavailable"}] + + [{role:"synthesis",reviewer:null,status:"passed",run_id:null, + evidence_status:"unavailable"}])')" + fi + fi + + result_tmp="$(mktemp "${result_file}.assurance-tmp.XXXXXX")" || { + printf 'Error: unable to create gate result temporary file beside: %s\n' \ + "$result_file" >&2 + return 1 + } + awk -v pointer="$ASSURANCE_POINTER" ' + /^---$/ { + fence++ + print + next + } + fence == 1 && /^gate_result_version:/ { + print "gate_result_version: pr_gate_result_v2" + print "gate_assurance: " pointer + next + } + fence == 1 && /^gate_assurance:/ { next } + { print } + ' "$result_file" > "$result_tmp" || { + rm -f -- "$result_tmp" + return 1 + } + result_sha="$(_gate_result_sha256_file "$result_tmp")" || { + rm -f -- "$result_tmp" + return 1 + } + + assurance_tmp="$(mktemp "${assurance_file}.tmp.XXXXXX")" || { + rm -f -- "$result_tmp" + printf 'Error: unable to create gate assurance temporary file beside: %s\n' \ + "$assurance_file" >&2 + return 1 + } + if ! jq -n \ + --arg final "$final" \ + --arg result_sha "$result_sha" \ + --arg repo_root "$WORK_DIR" --arg repo_identity "$GATE_BINDING_REPO_IDENTITY" \ + --arg base_commit "$GATE_BINDING_BASE_COMMIT" \ + --arg head_commit "$GATE_BINDING_HEAD_COMMIT" \ + --arg subject_fingerprint "$GATE_BINDING_SUBJECT_FINGERPRINT" \ + --arg tier_requested "$TIER_REQUESTED" --arg tier_resolved "$TIER_RESOLVED" \ + --arg evidence_floor "$TIER_EVIDENCE_FLOOR" \ + --arg mode_requested "$MODE_REQUESTED" --arg mode_resolved "$MODE_RESOLVED" \ + --arg topology "$MODE_TOPOLOGY" --arg synthesis "$MODE_SYNTHESIS" \ + --arg pass_requested "$PASS_KIND_REQUESTED" --arg pass_resolved "$PASS_KIND_RESOLVED" \ + --arg pass_scope "$PASS_SCOPE" --arg initial_result "$INITIAL_RESULT_RESOLVED" \ + --arg selected "$REVIEWERS" --arg skipped "$SKIPPED_WORDS" \ + --arg vocabulary "$ALL_REVIEWERS" \ + --arg reviewer_topology "$MODE_TOPOLOGY" \ + --arg independence_status "$independence_status" \ + --arg policy_source "$GATE_ASSURANCE_POLICY_SOURCE" \ + --arg attestation "$ASSURANCE_ATTESTATION_POINTER" \ + --argjson requested "$requested_json" --argjson outcomes "$outcomes_json" \ + --argjson implementation_isolated "$implementation_isolated" \ + --argjson per_reviewer_independent "$per_reviewer_independent" ' + { + kind:"gate_assurance_v2",schema_version:2, + result:{final:$final}, + bindings:{ + result_sha256:$result_sha, + repo_root:$repo_root, + repo_identity:$repo_identity, + base_commit:$base_commit, + head_commit:$head_commit, + subject_fingerprint:$subject_fingerprint + }, + coordinates:{ + tier:{requested:$tier_requested,resolved:$tier_resolved, + evidence_floor:$evidence_floor}, + mode:{requested:$mode_requested,resolved:$mode_resolved, + topology:$topology,synthesis:$synthesis}, + pass:{requested:$pass_requested,resolved:$pass_resolved,scope:$pass_scope, + initial_result:(if $initial_result == "" then null else $initial_result end)}, + coverage:{ + requested:$requested, + selected:($selected | split(" ") | map(select(length > 0))), + skipped:($skipped | split(" ") | map(select(length > 0))), + vocabulary:($vocabulary | split(" ") | map(select(length > 0))) + }, + independence:{ + implementation_context_isolated:$implementation_isolated, + reviewer_topology:$reviewer_topology, + per_reviewer_independent:$per_reviewer_independent, + evidence_status:$independence_status + } + }, + dispatch:{outcomes:$outcomes}, + provenance:{ + producer:"pr-gate.sh", + policy_source:$policy_source, + attestation:(if $attestation == "" then null else $attestation end) + } + }' > "$assurance_tmp"; then + rm -f -- "$assurance_tmp" "$result_tmp" + return 1 + fi + + _gate_assurance_destination_check "$assurance_file" || { + rm -f -- "$assurance_tmp" "$result_tmp" + return 1 + } + # Publish the sidecar before the v2 result that references it. A verifier + # racing this boundary sees either the original self-contained v1 result or + # the complete v2 pair; a host failure cannot strand a v2 result with a + # permanently missing sidecar. + mv -- "$assurance_tmp" "$assurance_file" || { + rm -f -- "$assurance_tmp" "$result_tmp" + return 1 + } + mv -- "$result_tmp" "$result_file" || { + rm -f -- "$result_tmp" + return 1 + } + gate_result_verify "$result_file" "" "machine assurance finalization" || return $? + + if [[ "$independence_status" == verified ]]; then + assurance_sha="$(_gate_result_sha256_file "$assurance_file")" || return $? + run_ids_json="$(jq -c '[.[].run_id]' <<<"$outcomes_json")" || return 1 + _gate_assurance_destination_check "$ASSURANCE_ATTESTATION_FILE" || return 1 + attestation_tmp="$(mktemp "${ASSURANCE_ATTESTATION_FILE}.tmp.XXXXXX")" || { + printf 'Error: unable to create protected gate assurance attestation\n' >&2 + return 1 + } + if ! jq -n \ + --arg result_sha "$result_sha" --arg assurance_sha "$assurance_sha" \ + --arg repo_root "$WORK_DIR" --arg repo_identity "$GATE_BINDING_REPO_IDENTITY" \ + --arg base_commit "$GATE_BINDING_BASE_COMMIT" \ + --arg head_commit "$GATE_BINDING_HEAD_COMMIT" \ + --arg subject_fingerprint "$GATE_BINDING_SUBJECT_FINGERPRINT" \ + --argjson run_ids "$run_ids_json" '{ + kind:"gate_assurance_attestation_v1", + schema_version:1, + result_sha256:$result_sha, + assurance_sha256:$assurance_sha, + repo_root:$repo_root, + repo_identity:$repo_identity, + base_commit:$base_commit, + head_commit:$head_commit, + subject_fingerprint:$subject_fingerprint, + run_ids:$run_ids + }' > "$attestation_tmp"; then + rm -f -- "$attestation_tmp" + return 1 + fi + mv -- "$attestation_tmp" "$ASSURANCE_ATTESTATION_FILE" || { + rm -f -- "$attestation_tmp" + return 1 + } + gate_assurance_authorization_verify "$result_file" "$assurance_file" \ + "$ASSURANCE_ATTESTATION_FILE" "$GATE_ASSURANCE_RUNS_FILE" + fi +} + SYNTHESIS_BRIEF="$BRIEF_DIR/pr-gate-${TIMESTAMP}-synthesis.md" BRIEF_FILES+=("$SYNTHESIS_BRIEF") @@ -1680,6 +2242,11 @@ _preflight_repo_identity() { printf '%s\n%s\n' "$WORK_DIR" "$remote" | _preflight_sha256_stream } +GATE_BINDING_SUBJECT_FINGERPRINT="$(_preflight_tree_fingerprint)" || exit 2 +GATE_BINDING_REPO_IDENTITY="$(_preflight_repo_identity)" || exit 2 +GATE_BINDING_BASE_COMMIT="$(git rev-parse "${BASE}^{commit}")" || exit 2 +GATE_BINDING_HEAD_COMMIT="$(git rev-parse "${HEAD_REF}^{commit}")" || exit 2 + if [[ "$SKIP_PREFLIGHT_TESTS" != "true" && -n "$TEST_CMD_OVERRIDE" ]]; then # pr-gate.sh is designed to be copied standalone into any repo (copy-mode -- # see the file header), so it must not hardcode any repo-specific test @@ -1694,10 +2261,10 @@ if [[ "$SKIP_PREFLIGHT_TESTS" != "true" && -n "$TEST_CMD_OVERRIDE" ]]; then PREFLIGHT_EVIDENCE_PATH="$WORK_DIR/.gate-results/preflight-evidence-${TIMESTAMP}.json" PREFLIGHT_RICH_RESULT_PATH="$WORK_DIR/.gate-results/preflight-rich-result-${TIMESTAMP}.json" _preflight_command_digest="$(printf '%s' "$TEST_CMD_OVERRIDE" | _preflight_sha256_stream)" || exit 2 - _preflight_before="$(_preflight_tree_fingerprint)" || exit 2 - _preflight_repo_id="$(_preflight_repo_identity)" || exit 2 - _preflight_base_commit="$(git rev-parse "${BASE}^{commit}")" || exit 2 - _preflight_head_commit="$(git rev-parse "${HEAD_REF}^{commit}")" || exit 2 + _preflight_before="$GATE_BINDING_SUBJECT_FINGERPRINT" + _preflight_repo_id="$GATE_BINDING_REPO_IDENTITY" + _preflight_base_commit="$GATE_BINDING_BASE_COMMIT" + _preflight_head_commit="$GATE_BINDING_HEAD_COMMIT" _preflight_started="$(date -u +'%Y-%m-%dT%H:%M:%SZ')" say 'pr-gate: running pre-flight test suite (timeout %ss, command sha256:%s)\n' \ "$TEST_TIMEOUT" "${_preflight_command_digest:0:12}" @@ -1881,7 +2448,7 @@ _write_preflight_failure_result() { gate_result_version: pr_gate_result_v1 final: NO-GO tier: ${TIER} -mode: preflight-fail-fast +mode: ${MODE_RESOLVED} most_severe: block reviewers: ${reviewer_lines}escalation: @@ -2778,6 +3345,12 @@ elif [[ -x "$_POST_GATE_HOOK" ]]; then fi fi +# Replace the executor-authored staging frontmatter with a v2 pointer and write +# the machine-owned assurance sidecar only after every deterministic rewrite +# and explicitly enabled post-gate hook is complete. The shared verifier then +# checks result/pointer/envelope parity before publication or relocation. +gate_finalize_assurance "$OUTPUT_FILE" "$ASSURANCE_FILE" || exit 1 + # ── Relocate result to run dir (post-verification) ─────────────────────────── # OUTPUT_FILE was written by the executor in WORK_DIR (workspace-write sandbox # constraint). Now that it is verified, move it (and any parallel reviewer outputs, diff --git a/runtime/lib/gate-result-verify.sh b/runtime/lib/gate-result-verify.sh index 62b237ed..2f2a1c88 100644 --- a/runtime/lib/gate-result-verify.sh +++ b/runtime/lib/gate-result-verify.sh @@ -1,75 +1,343 @@ #!/usr/bin/env bash -# Shared gate-result integrity verification. +# Shared gate-result integrity and assurance verification. # -# A gate result file (pr_gate_result_v1) is only trustworthy when it is -# structurally complete: a non-empty file carrying exactly one parseable -# `Final: GO|NO-GO` verdict whose value agrees with the YAML frontmatter -# `final:` field. The single-session route, the parallel synthesis route, and -# `pmctl gate verify` all enforce the SAME contract, so the checks live here -# once instead of being re-encoded per call site. -# -# This is the authority the gate uses to turn "the executor exited 0" into -# "the executor actually produced a verdict" -- the seam that catches a session -# that exits clean without writing the result (the 0-byte result failure mode). -# -# pr-gate.sh carries an inline copy of gate_result_verify as a fallback for -# copy-mode (running pr-gate.sh standalone without runtime/lib/ co-located); the -# two MUST stay in sync. The copy-mode regression test exercises the fallback. +# pr_gate_result_v1 remains valid legacy structural evidence. It proves only +# frontmatter/body verdict parity and is reported as assurance=unavailable. +# pr_gate_result_v2 points at a sibling assurance JSON envelope owned by the +# gate shell. Legacy gate_assurance_v1 remains readable but non-authorizing; +# gate_assurance_v2 adds subject/result bindings and protected attestation. -# gate_result_verify [expected_final] [route_label] -# Returns 0 when is a structurally valid gate result. On the first -# failed check it prints a specific diagnostic to stderr and returns 1: -# - file exists and is non-empty -# - exactly one `^Final: (GO|NO-GO)$` line (plain text, no markdown emphasis) -# - YAML frontmatter carries a `final:` field -# - frontmatter `final:` equals the body `Final:` value -# - when [expected_final] is non-empty, the body `Final:` must equal it (the -# parallel route passes the shell-computed verdict so a synthesis that -# contradicts it is rejected as manipulated/corrupt) -# [route_label] (default "gate") is woven into the not-produced / contradiction -# diagnostics so each call site reports in its own vocabulary. -gate_result_verify() { +# gate_result_verdict_verify [expected_final] [route_label] +gate_result_verdict_verify() { local result_file=${1-} expected_final=${2-} route_label=${3-gate} local final_count frontmatter_final body_final [[ $# -ge 1 && $# -le 3 ]] || { - printf 'gate-result-verify: gate_result_verify expects [expected_final] [route_label]\n' >&2 + printf 'gate-result-verify: gate_result_verdict_verify expects [expected_final] [route_label]\n' >&2 return 2 } - if [[ ! -s "$result_file" ]]; then printf 'Error: %s did not produce the result file: %s\n' "$route_label" "$result_file" >&2 printf 'Gate aborted -- the executor session may have exited 0 without writing a verdict.\n' >&2 return 1 fi - final_count=$(grep -cE '^Final: (GO|NO-GO)$' "$result_file" || true) if [[ "$final_count" -ne 1 ]]; then printf 'Error: gate result file must contain exactly one Final: GO/NO-GO line (found %d): %s\n' \ "$final_count" "$result_file" >&2 return 1 fi - frontmatter_final=$(awk 'BEGIN{s=0} /^---$/ { if (s == 0) { s=1; next } else if (s == 1) { exit } } s && $1 == "final:" { print $2; exit }' "$result_file") if [[ -z "$frontmatter_final" ]]; then printf 'Error: gate result YAML frontmatter missing required field: final: (%s)\n' "$result_file" >&2 return 1 fi - body_final=$(grep -E '^Final: (GO|NO-GO)$' "$result_file" | awk '{print $2}') if [[ "$frontmatter_final" != "$body_final" ]]; then printf 'Error: frontmatter final: (%s) does not match body Final: (%s) in gate result: %s\n' \ "$frontmatter_final" "$body_final" "$result_file" >&2 return 1 fi - if [[ -n "$expected_final" && "$body_final" != "$expected_final" ]]; then printf 'Error: %s verdict (%s) contradicts shell-computed verdict (%s) -- gate result may have been manipulated: %s\n' \ "$route_label" "$body_final" "$expected_final" "$result_file" >&2 return 1 fi +} + +_gate_result_frontmatter_value() { + local result_file="$1" key="$2" + awk -v wanted="$key" ' + BEGIN{s=0} + /^---$/ { if (s == 0) { s=1; next } else if (s == 1) { exit } } + s && $1 == wanted ":" { print $2; exit } + ' "$result_file" +} + +# The core JSON Schema owns portable envelope shape. This runtime predicate owns +# cross-artifact and semantic claim consistency that JSON Schema cannot establish +# from the Markdown result. The standalone fallback is an exact generated-style +# copy guarded by test_inline_fallback_matches_lib. +# +# gate_assurance_verify +gate_assurance_verify() { + local result_file="$1" assurance_file="$2" body_final="$3" + local markdown_tier markdown_mode result_sha assurance_kind + command -v jq >/dev/null 2>&1 || { + printf 'Error: gate assurance verification requires jq\n' >&2 + return 2 + } + if [[ ! -s "$assurance_file" ]]; then + printf 'Error: gate assurance sidecar missing or empty: %s\n' "$assurance_file" >&2 + return 1 + fi + markdown_tier="$(_gate_result_frontmatter_value "$result_file" tier)" + markdown_mode="$(_gate_result_frontmatter_value "$result_file" mode)" + assurance_kind="$(jq -r '.kind // empty' "$assurance_file" 2>/dev/null)" + if [[ "$assurance_kind" == gate_assurance_v1 ]]; then + jq -e --arg final "$body_final" --arg markdown_tier "$markdown_tier" \ + --arg markdown_mode "$markdown_mode" ' + .kind == "gate_assurance_v1" and .schema_version == 1 and + .result.final == $final and + .coordinates.tier.resolved == $markdown_tier and + .coordinates.mode.resolved == $markdown_mode + ' "$assurance_file" >/dev/null || { + printf 'Error: legacy gate assurance sidecar failed claim verification: %s\n' \ + "$assurance_file" >&2 + return 1 + } + GATE_ASSURANCE_BOUND=false + export GATE_ASSURANCE_BOUND + return 0 + fi + result_sha="$(_gate_result_sha256_file "$result_file")" || return $? + jq -e --arg final "$body_final" --arg result_sha "$result_sha" \ + --arg markdown_tier "$markdown_tier" \ + --arg markdown_mode "$markdown_mode" ' + def only_keys($allowed): + type == "object" and ((keys_unsorted - $allowed) | length) == 0; + def strings_unique: + type == "array" and all(.[]; type == "string" and length > 0) and + (length == (unique | length)); + def same_set($a; $b): ($a | sort) == ($b | sort); + only_keys(["kind","schema_version","result","bindings","coordinates", + "dispatch","provenance"]) and + (.result | only_keys(["final"])) and + (.bindings | only_keys(["result_sha256","repo_root","repo_identity", + "base_commit","head_commit","subject_fingerprint"])) and + (.coordinates | only_keys(["tier","mode","pass","coverage","independence"])) and + (.coordinates.tier | only_keys(["requested","resolved","evidence_floor"])) and + (.coordinates.mode | only_keys(["requested","resolved","topology","synthesis"])) and + (.coordinates.pass | only_keys(["requested","resolved","scope","initial_result"])) and + (.coordinates.coverage | + only_keys(["requested","selected","skipped","vocabulary"])) and + (.coordinates.independence | + only_keys(["implementation_context_isolated","reviewer_topology", + "per_reviewer_independent","evidence_status"])) and + (.dispatch | only_keys(["outcomes"])) and + (all(.dispatch.outcomes[]; + only_keys(["role","reviewer","status","run_id","evidence_status"]))) and + (.provenance | only_keys(["producer","policy_source","attestation"])) and + .kind == "gate_assurance_v2" and .schema_version == 2 and + .result.final == $final and + .bindings.result_sha256 == $result_sha and + (.bindings.repo_root | type == "string" and startswith("/")) and + (.bindings.repo_identity | test("^[a-f0-9]{64}$")) and + (.bindings.base_commit | test("^[a-f0-9]{40}$")) and + (.bindings.head_commit | test("^[a-f0-9]{40}$")) and + (.bindings.subject_fingerprint | test("^[a-f0-9]{64}$")) and + .coordinates.tier.resolved == $markdown_tier and + .coordinates.mode.resolved == $markdown_mode and + .provenance.producer == "pr-gate.sh" and + (.provenance.policy_source | + IN("canonical","generated-snapshot","mixed")) and + (.coordinates.tier.evidence_floor | type == "string" and length > 0) and + (.coordinates.tier.requested == "auto" or + (.coordinates.tier.requested == .coordinates.tier.resolved and + (.coordinates.tier.requested | IN("express","standard","full")))) and + (.coordinates.tier.resolved | IN("express","standard","full")) and + (.coordinates.mode.requested | IN("default","sequential","parallel")) and + (.coordinates.mode.resolved | IN("sequential","parallel")) and + (.coordinates.mode.requested == "default" or + .coordinates.mode.requested == .coordinates.mode.resolved) and + ((.coordinates.mode.resolved == "sequential" and + .coordinates.mode.topology == "combined-session" and + .coordinates.mode.synthesis == "inline") or + (.coordinates.mode.resolved == "parallel" and + .coordinates.mode.topology == "per-reviewer-sessions" and + .coordinates.mode.synthesis == "separate-session")) and + (.coordinates.pass.requested | IN("initial","targeted")) and + (.coordinates.pass.resolved | IN("initial","targeted")) and + .coordinates.pass.requested == .coordinates.pass.resolved and + ((.coordinates.pass.resolved == "initial" and + .coordinates.pass.scope == "comprehensive" and + .coordinates.pass.initial_result == null) or + (.coordinates.pass.resolved == "targeted" and + .coordinates.pass.scope == "remediation-delta" and + (.coordinates.pass.initial_result | type == "string" and length > 0))) and + (.coordinates.coverage.vocabulary | strings_unique) and + (.coordinates.coverage.selected | strings_unique) and + (.coordinates.coverage.skipped | strings_unique) and + ((.coordinates.coverage.selected + .coordinates.coverage.skipped) | strings_unique) and + same_set(.coordinates.coverage.selected + .coordinates.coverage.skipped; + .coordinates.coverage.vocabulary) and + (.coordinates.coverage.requested == null or + ((.coordinates.coverage.requested | strings_unique) and + same_set(.coordinates.coverage.requested; + .coordinates.coverage.selected))) and + (.dispatch.outcomes | type == "array") and + (all(.dispatch.outcomes[]; + (.role | IN("combined","reviewer","synthesis","preflight")) and + (if .role == "reviewer" + then (.reviewer | type == "string" and length > 0) + else .reviewer == null + end) and + (.status | IN("passed","failed","skipped")) and + (.evidence_status | IN("verified","unavailable","unverified")) and + (.run_id == null or + (.run_id | type == "string" and test("^run-[A-Za-z0-9]+-[A-Za-z0-9]+$"))))) and + (if .coordinates.mode.resolved == "parallel" and + ([.dispatch.outcomes[] | select(.role == "preflight")] | length) == 0 + then + (.dispatch.outcomes | length) == + ((.coordinates.coverage.selected | length) + 1) and + (all(.dispatch.outcomes[]; + (.role == "reviewer" or .role == "synthesis"))) and + ([.dispatch.outcomes[] | select(.role == "reviewer") | .reviewer] | sort) == + (.coordinates.coverage.selected | sort) and + ([.dispatch.outcomes[] | select(.role == "synthesis")] | length) == 1 + elif .coordinates.mode.resolved == "sequential" and + ([.dispatch.outcomes[] | select(.role == "preflight")] | length) == 0 + then + (.dispatch.outcomes | length) == 1 and + .dispatch.outcomes[0].role == "combined" + else + (.dispatch.outcomes | length) == 1 and + .dispatch.outcomes[0].role == "preflight" and + .dispatch.outcomes[0].status == "failed" + end) and + (.coordinates.independence.evidence_status | + IN("verified","unavailable","unverified")) and + (if .coordinates.independence.evidence_status == "verified" + then (.provenance.attestation | + type == "string" and + test("^gate-assurance-[0-9]{8}-[0-9]{6}\\.attestation\\.json$")) + else .provenance.attestation == null + end) and + .coordinates.independence.reviewer_topology == + .coordinates.mode.topology and + (if .coordinates.independence.evidence_status == "verified" + then + .coordinates.independence.implementation_context_isolated == true and + (all(.dispatch.outcomes[]; .evidence_status == "verified" and .run_id != null)) and + ([.dispatch.outcomes[].run_id] | length == (unique | length)) and + (if .coordinates.mode.resolved == "parallel" + then .coordinates.independence.per_reviewer_independent == true + else .coordinates.independence.per_reviewer_independent == false + end) + else + .coordinates.independence.implementation_context_isolated != true and + .coordinates.independence.per_reviewer_independent != true and + (all(.dispatch.outcomes[]; .evidence_status != "verified")) + end) + ' "$assurance_file" >/dev/null || { + printf 'Error: gate assurance sidecar failed structural/claim verification: %s\n' \ + "$assurance_file" >&2 + return 1 + } + GATE_ASSURANCE_BOUND=true + export GATE_ASSURANCE_BOUND +} + +_gate_result_sha256_file() { + local file="$1" digest="" + if command -v sha256sum >/dev/null 2>&1 \ + && digest="$(sha256sum -- "$file" 2>/dev/null | awk '{print $1}')" \ + && [[ -n "$digest" ]]; then + printf '%s\n' "$digest" + return 0 + fi + if command -v shasum >/dev/null 2>&1 \ + && digest="$(shasum -a 256 -- "$file" 2>/dev/null | awk '{print $1}')" \ + && [[ -n "$digest" ]]; then + printf '%s\n' "$digest" + return 0 + fi + printf 'Error: no sha256sum or shasum found -- cannot verify gate assurance binding\n' >&2 + return 2 +} - return 0 +# gate_assurance_authorization_verify +# Validates the protected producer attestation and resolves every claimed run ID +# to the latest canonical terminal record for the same gate run and repository. +gate_assurance_authorization_verify() { + local result_file="$1" assurance_file="$2" attestation_file="$3" runs_file="$4" + local result_sha assurance_sha run_root + [[ -s "$attestation_file" && -s "$runs_file" ]] || { + printf 'Error: verified gate assurance requires protected attestation and canonical run records\n' >&2 + return 1 + } + result_sha="$(_gate_result_sha256_file "$result_file")" || return $? + assurance_sha="$(_gate_result_sha256_file "$assurance_file")" || return $? + run_root="$(cd "$(dirname "$attestation_file")" && pwd -P)" || return 1 + jq -e --arg result_sha "$result_sha" --arg assurance_sha "$assurance_sha" \ + --slurpfile assurance "$assurance_file" ' + $assurance[0] as $a | + .kind == "gate_assurance_attestation_v1" and .schema_version == 1 and + .result_sha256 == $result_sha and .assurance_sha256 == $assurance_sha and + .repo_root == $a.bindings.repo_root and + .repo_identity == $a.bindings.repo_identity and + .base_commit == $a.bindings.base_commit and + .head_commit == $a.bindings.head_commit and + .subject_fingerprint == $a.bindings.subject_fingerprint and + ([.run_ids[]] | sort) == + ([$a.dispatch.outcomes[].run_id] | sort) + ' "$attestation_file" >/dev/null || { + printf 'Error: gate assurance protected attestation mismatch: %s\n' \ + "$attestation_file" >&2 + return 1 + } + jq -s -e --slurpfile assurance "$assurance_file" \ + --arg run_root "$run_root" ' + $assurance[0] as $a | + . as $records | + all($a.dispatch.outcomes[].run_id; + . as $id | + ([$records[] | select(.id == $id)] | last) as $record | + $record != null and $record.state == "ok" and $record.exit_code == 0 and + $record.working_dir == $a.bindings.repo_root and + ($record.trace_path | type == "string" and + startswith($run_root + "/.agent-trace/"))) + ' "$runs_file" >/dev/null || { + printf 'Error: gate assurance dispatch evidence does not match canonical run records\n' >&2 + return 1 + } +} + +# gate_result_verify [expected_final] [route_label] +gate_result_verify() { + local result_file=${1-} expected_final=${2-} route_label=${3-gate} + local version pointer result_parent assurance_file body_final + [[ $# -ge 1 && $# -le 3 ]] || { + printf 'gate-result-verify: gate_result_verify expects [expected_final] [route_label]\n' >&2 + return 2 + } + gate_result_verdict_verify "$result_file" "$expected_final" "$route_label" || return $? + version="$(_gate_result_frontmatter_value "$result_file" gate_result_version)" + case "$version" in + pr_gate_result_v1) + GATE_RESULT_ASSURANCE=unavailable + unset GATE_RESULT_ASSURANCE_FILE + export GATE_RESULT_ASSURANCE + return 0 + ;; + pr_gate_result_v2) + pointer="$(_gate_result_frontmatter_value "$result_file" gate_assurance)" + if [[ -z "$pointer" || "$pointer" == */* || "$pointer" == "." || "$pointer" == ".." \ + || ! "$pointer" =~ ^[A-Za-z0-9._-]+\.json$ ]]; then + printf 'Error: pr_gate_result_v2 requires a bounded sibling gate_assurance pointer: %s\n' \ + "$result_file" >&2 + return 1 + fi + result_parent="$(cd "$(dirname "$result_file")" && pwd -P)" || return 1 + assurance_file="$result_parent/$pointer" + body_final=$(grep -E '^Final: (GO|NO-GO)$' "$result_file" | awk '{print $2}') + gate_assurance_verify "$result_file" "$assurance_file" "$body_final" || return $? + if [[ "${GATE_ASSURANCE_BOUND:-false}" == true ]]; then + GATE_RESULT_ASSURANCE=verified + else + GATE_RESULT_ASSURANCE=unavailable + fi + GATE_RESULT_ASSURANCE_FILE="$assurance_file" + export GATE_RESULT_ASSURANCE GATE_RESULT_ASSURANCE_FILE + ;; + *) + printf 'Error: unsupported or missing gate_result_version in gate result: %s\n' \ + "$result_file" >&2 + return 1 + ;; + esac } -export -f gate_result_verify +export -f gate_result_verdict_verify gate_assurance_verify \ + gate_assurance_authorization_verify gate_result_verify diff --git a/runtime/lib/pmctl-gate.sh b/runtime/lib/pmctl-gate.sh index b2fdb89e..17e6b7c4 100644 --- a/runtime/lib/pmctl-gate.sh +++ b/runtime/lib/pmctl-gate.sh @@ -64,6 +64,51 @@ _pmctl_gate_operation_ensure_loaded() { && _pmctl_operation_ensure_loaded "$repo_root" } +# A v2 result and its bound sidecar cannot be renamed into place as one +# filesystem operation. If a verifier races the producer between those +# renames, wait briefly for the sidecar (and, for verified independence, +# its protected attestation) instead of turning a valid in-flight +# finalization into a permanent false negative. +_pmctl_gate_wait_for_assurance_publication() { + local result_file="$1" version pointer result_parent assurance_file + local assurance_kind evidence_status attestation_pointer run_root + local attempt=0 max_attempts=20 + + version="$(_gate_result_frontmatter_value "$result_file" gate_result_version)" + [[ "$version" == pr_gate_result_v2 ]] || return 0 + result_parent="$(cd "$(dirname "$result_file")" 2>/dev/null && pwd -P)" || return 0 + # Only canonical run-layout results have an asynchronous producer + # finalization lifecycle. Ad-hoc and copy-mode results fail immediately. + [[ "$(basename "$result_parent")" == ".gate-results" ]] || return 0 + pointer="$(_gate_result_frontmatter_value "$result_file" gate_assurance)" + if [[ -z "$pointer" || "$pointer" == */* || "$pointer" == "." || "$pointer" == ".." \ + || ! "$pointer" =~ ^[A-Za-z0-9._-]+\.json$ ]]; then + return 0 + fi + assurance_file="$result_parent/$pointer" + run_root="$(dirname "$result_parent")" + + while (( attempt < max_attempts )); do + if [[ -s "$assurance_file" ]]; then + assurance_kind="$(jq -r '.kind // empty' "$assurance_file" 2>/dev/null)" + evidence_status="$(jq -r \ + '.coordinates.independence.evidence_status // "unavailable"' \ + "$assurance_file" 2>/dev/null || printf 'unavailable')" + if [[ "$assurance_kind" != gate_assurance_v2 || "$evidence_status" != verified ]]; then + return 0 + fi + attestation_pointer="$(jq -r '.provenance.attestation // empty' \ + "$assurance_file" 2>/dev/null)" + if [[ -n "$attestation_pointer" && "$attestation_pointer" != */* \ + && -s "$run_root/$attestation_pointer" ]]; then + return 0 + fi + fi + attempt=$((attempt + 1)) + sleep 0.1 + done +} + pmctl_gate_run() { local repo_root="$1"; shift @@ -659,6 +704,9 @@ pmctl_gate_verify() { return 2 fi local result_file="$1" + local evidence_status attestation_pointer result_parent run_root run_id + local attestation_file runs_file canonical_repo_root assurance_repo_root + local canonical_run_root if ! declare -F gate_result_verify >/dev/null; then local lib="$repo_root/runtime/lib/gate-result-verify.sh" @@ -670,8 +718,61 @@ pmctl_gate_verify() { . "$lib" fi + _pmctl_gate_wait_for_assurance_publication "$result_file" if gate_result_verify "$result_file"; then + evidence_status="$(jq -r '.coordinates.independence.evidence_status // "unavailable"' \ + "${GATE_RESULT_ASSURANCE_FILE:-/dev/null}" 2>/dev/null || printf 'unavailable')" + if [[ "${GATE_RESULT_ASSURANCE:-unavailable}" == verified \ + && "$evidence_status" == verified ]]; then + attestation_pointer="$(jq -r '.provenance.attestation // empty' \ + "$GATE_RESULT_ASSURANCE_FILE" 2>/dev/null)" + if [[ -z "$attestation_pointer" || "$attestation_pointer" == */* \ + || ! "$attestation_pointer" =~ ^gate-assurance-[0-9]{8}-[0-9]{6}\.attestation\.json$ ]]; then + printf 'Error: verified gate assurance requires a bounded protected attestation pointer\n' >&2 + return 1 + fi + result_parent="$(cd "$(dirname "$result_file")" && pwd -P)" || return 1 + if [[ "$(basename "$result_parent")" != ".gate-results" ]]; then + printf 'Error: verified gate assurance result is outside a canonical gate run directory\n' >&2 + return 1 + fi + run_root="$(dirname "$result_parent")" + run_id="$(basename "$run_root")" + canonical_repo_root="$(git rev-parse --show-toplevel 2>/dev/null)" || { + printf 'Error: verified gate assurance requires invocation from its repository\n' >&2 + return 1 + } + canonical_repo_root="$(cd "$canonical_repo_root" && pwd -P)" || return 1 + assurance_repo_root="$(jq -r '.bindings.repo_root // empty' \ + "$GATE_RESULT_ASSURANCE_FILE" 2>/dev/null)" + if [[ -z "$assurance_repo_root" || ! -d "$assurance_repo_root" \ + || "$(cd "$assurance_repo_root" && pwd -P)" != "$canonical_repo_root" ]]; then + printf 'Error: verified gate assurance repository binding does not match the invoking repository\n' >&2 + return 1 + fi + if ! _pmctl_gate_ensure_run_dir_fn "$repo_root"; then + printf 'Error: verified gate assurance requires canonical state-path resolution\n' >&2 + return 2 + fi + canonical_run_root="$( + cd "$canonical_repo_root" || exit 1 + _SW_REPO_ROOT="$canonical_repo_root" sw_project_run_dir "$run_id" + )" || return 1 + if [[ ! -d "$canonical_run_root" \ + || "$(cd "$canonical_run_root" && pwd -P)" != "$run_root" ]]; then + printf 'Error: verified gate assurance result is outside the invoking repository canonical state partition\n' >&2 + return 1 + fi + attestation_file="$run_root/$attestation_pointer" + runs_file="$(dirname "$(dirname "$canonical_run_root")")/runs.jsonl" + gate_assurance_authorization_verify "$result_file" "$GATE_RESULT_ASSURANCE_FILE" \ + "$attestation_file" "$runs_file" || return $? + fi printf 'gate result OK: %s\n' "$result_file" + printf 'assurance: %s\n' "${GATE_RESULT_ASSURANCE:-unavailable}" + if [[ -n "${GATE_RESULT_ASSURANCE_FILE:-}" ]]; then + printf 'assurance file: %s\n' "$GATE_RESULT_ASSURANCE_FILE" + fi return 0 fi return 1 diff --git a/skills/pr-gate-review/SKILL.md b/skills/pr-gate-review/SKILL.md index 6514fa66..462bb167 100644 --- a/skills/pr-gate-review/SKILL.md +++ b/skills/pr-gate-review/SKILL.md @@ -31,9 +31,20 @@ implement → pr-gate → fix NO-GO → push → PR). ## Reading the result -The result file carries `pr_gate_result_v1` frontmatter with `final: GO|NO-GO` -and per-reviewer verdicts. The `Final: GO|NO-GO` line is the parser-significant -one (plain text, exact shape). +The result file carries `pr_gate_result_v2` frontmatter with `final: GO|NO-GO`, +per-reviewer verdicts, and a bounded pointer to its sibling +`gate_assurance_v2` JSON envelope. The `Final: GO|NO-GO` line is the +parser-significant one (plain text, exact shape). Run +`pmctl gate verify ` from the reviewed repository before consuming +assurance claims; legacy +`pr_gate_result_v1` files and unbound v1 envelopes verify only as +`assurance: unavailable`. A standalone +copy-mode v2 result may also carry `evidence_status: unavailable` inside its +valid envelope; treat its verdict as structurally valid without inferring +implementation isolation or independent reviewer sessions. Repo-layout +independence is authoritative only when verification also confirms the +protected producer attestation, the invoking repository's canonical state +partition, and every claimed canonical run record. - **NO-GO** (a reviewer returned `block`): fix the blocking finding. Per project convention, clear **every** finding (high/med/low/advise) on a NO-GO, not just diff --git a/tests/bin/run-tests.sh b/tests/bin/run-tests.sh index 5f4fc7c4..1b8c44b1 100755 --- a/tests/bin/run-tests.sh +++ b/tests/bin/run-tests.sh @@ -215,6 +215,8 @@ map_path() { add_suite test-pmctl-task; behavioral=1 ;; core/schema/preflight-evidence.schema.json) add_suite test-pr-gate; behavioral=1 ;; + tools/generate-gate-result-verifier-fallback.sh) + add_suite test-pr-gate; behavioral=1 ;; core/policy/gate-tiers.tsv|core/policy/gate-modes.tsv|core/policy/gate-pass-kinds.tsv) add_suite test-pr-gate; add_suite test-pr-gate-profile; behavioral=1 ;; runtime/lib/pmctl-config.sh) diff --git a/tests/shell/test-core-schemas.sh b/tests/shell/test-core-schemas.sh index 0c6be042..7a3816e7 100755 --- a/tests/shell/test-core-schemas.sh +++ b/tests/shell/test-core-schemas.sh @@ -741,6 +741,88 @@ case_preflight_reusable_evidence_requires_fingerprint() { rm -f "$tmpf" } +_gate_assurance_valid_instance() { + jq -n '{ + kind:"gate_assurance_v2", + schema_version:2, + result:{final:"GO"}, + bindings:{ + result_sha256:("a" * 64), + repo_root:"/tmp/repo", + repo_identity:("b" * 64), + base_commit:("c" * 40), + head_commit:("d" * 40), + subject_fingerprint:("e" * 64) + }, + coordinates:{ + tier:{requested:"standard",resolved:"standard",evidence_floor:"critic plus QA"}, + mode:{ + requested:"sequential", + resolved:"sequential", + topology:"combined-session", + synthesis:"inline" + }, + pass:{ + requested:"initial", + resolved:"initial", + scope:"comprehensive", + initial_result:null + }, + coverage:{ + requested:null, + selected:["critic","qa-tester"], + skipped:["security"], + vocabulary:["critic","qa-tester","security"] + }, + independence:{ + implementation_context_isolated:null, + reviewer_topology:"combined-session", + per_reviewer_independent:null, + evidence_status:"unavailable" + } + }, + dispatch:{ + outcomes:[{ + role:"combined", + reviewer:null, + status:"passed", + run_id:null, + evidence_status:"unavailable" + }] + }, + provenance:{producer:"pr-gate.sh",policy_source:"canonical",attestation:null} + }' +} + +case_gate_assurance_valid_instance() { + local name="gate-assurance: canonical sequential envelope validates" + should_run "$name" || return 0 + local schema_file="$CORE_DIR/schema/gate-assurance.schema.json" tmpf + tmpf="$(mktemp /tmp/gate-assurance-valid-XXXXXX.json)" + _gate_assurance_valid_instance > "$tmpf" + if jsonschema -i "$tmpf" "$schema_file" >/dev/null 2>&1; then + pass "$name" + else + fail "$name" "schema rejected a canonical sequential assurance envelope" + fi + rm -f "$tmpf" +} + +case_gate_assurance_invalid_outcome_rejected() { + local name="gate-assurance: unknown dispatch status is rejected" + should_run "$name" || return 0 + local schema_file="$CORE_DIR/schema/gate-assurance.schema.json" tmpf + tmpf="$(mktemp /tmp/gate-assurance-invalid-XXXXXX.json)" + _gate_assurance_valid_instance | + jq '.dispatch.outcomes[0].status = "timed-out"' > "$tmpf" + if jsonschema -i "$tmpf" "$schema_file" >/dev/null 2>&1; then + fail "$name" "schema accepted an outcome status outside the contract" + else + pass "$name" + fi + rm -f "$tmpf" +} + case_context_pack_v1_still_valid case_context_pack_v2_new_fields_valid case_context_pack_memory_source_domain_valid @@ -748,5 +830,7 @@ case_context_pack_invalid_source_domain_rejected case_context_pack_invalid_trust_level_rejected case_preflight_basic_evidence_needs_no_git_provenance case_preflight_reusable_evidence_requires_fingerprint +case_gate_assurance_valid_instance +case_gate_assurance_invalid_outcome_rejected th_summary diff --git a/tests/shell/test-pmctl-gate.sh b/tests/shell/test-pmctl-gate.sh index 7c60c331..f39d33f0 100755 --- a/tests/shell/test-pmctl-gate.sh +++ b/tests/shell/test-pmctl-gate.sh @@ -22,6 +22,13 @@ th_init "$@" # deterministic and never collide with a real gate run on this host. _GATE_CLI_XDG_RUNTIME_DIR="$tmp_root/gate-cli-xdg-runtime" mkdir -p "$_GATE_CLI_XDG_RUNTIME_DIR" && chmod 700 "$_GATE_CLI_XDG_RUNTIME_DIR" +_GATE_VERIFY_REPO="$tmp_root/gate-verify-repo" +_GATE_VERIFY_STATE_ROOT="$tmp_root/gate-verify-state" +mkdir -p "$_GATE_VERIFY_REPO" "$_GATE_VERIFY_STATE_ROOT" +git -C "$_GATE_VERIFY_REPO" init -q + +# shellcheck source=runtime/lib/state-paths.sh +. "$REPO_ROOT/runtime/lib/state-paths.sh" # --------------------------------------------------------------------------- # Helpers @@ -394,6 +401,132 @@ Final: ${body_final} RESULT } +_mk_gate_result_v2() { + local path="$1" result_sha + _mk_gate_result "$path" GO + sed -i \ + -e 's/^gate_result_version: pr_gate_result_v1$/gate_result_version: pr_gate_result_v2/' \ + -e '/^gate_result_version:/a gate_assurance: result.md.assurance.json' \ + "$path" + result_sha="$(sha256sum "$path" | awk '{print $1}')" + jq -n --arg result_sha "$result_sha" '{ + kind:"gate_assurance_v2", + schema_version:2, + result:{final:"GO"}, + bindings:{ + result_sha256:$result_sha, + repo_root:"/tmp/repo", + repo_identity:("b" * 64), + base_commit:("c" * 40), + head_commit:("d" * 40), + subject_fingerprint:("e" * 64) + }, + coordinates:{ + tier:{requested:"auto",resolved:"express",evidence_floor:"reviewer-verdicts"}, + mode:{requested:"default",resolved:"sequential",topology:"combined-session",synthesis:"inline"}, + pass:{requested:"initial",resolved:"initial",scope:"comprehensive",initial_result:null}, + coverage:{requested:null,selected:["critic"],skipped:["qa-tester"], + vocabulary:["critic","qa-tester"]}, + independence:{implementation_context_isolated:null, + reviewer_topology:"combined-session",per_reviewer_independent:null, + evidence_status:"unavailable"} + }, + dispatch:{outcomes:[{role:"combined",reviewer:null,status:"passed", + run_id:null,evidence_status:"unavailable"}]}, + provenance:{producer:"pr-gate.sh",policy_source:"canonical",attestation:null} + }' > "${path}.assurance.json" +} + +_mk_gate_result_v2_verified() { + local path="$1" bound_repo="${2:-/tmp/repo}" sidecar="${1}.assurance.json" + local result_parent run_root project_dir attestation assurance_sha + _mk_gate_result_v2 "$path" + result_parent="$(dirname "$path")" + run_root="$(dirname "$result_parent")" + project_dir="$(dirname "$(dirname "$run_root")")" + attestation="$run_root/gate-assurance-20260727-000000.attestation.json" + mkdir -p "$run_root/.agent-trace" "$project_dir" + printf 'trace\n' > "$run_root/.agent-trace/test.last" + jq --arg bound_repo "$bound_repo" ' + .bindings.repo_root = $bound_repo | + .coordinates.independence = { + implementation_context_isolated:true, + reviewer_topology:"combined-session", + per_reviewer_independent:false, + evidence_status:"verified" + } | + .dispatch.outcomes = [{ + role:"combined",reviewer:null,status:"passed", + run_id:"run-20260727T000000Z-aaaaaa",evidence_status:"verified" + }] | + .provenance.attestation = "gate-assurance-20260727-000000.attestation.json" + ' "$sidecar" > "${sidecar}.tmp" + mv "${sidecar}.tmp" "$sidecar" + jq -nc --arg trace "$run_root/.agent-trace/test.last" \ + --arg bound_repo "$bound_repo" '{ + schema_version:3,id:"run-20260727T000000Z-aaaaaa",task_id:"UNKN-0", + executor:"codex",state:"ok",exit_code:0,model:"default", + brief_file:"/tmp/brief.md",working_dir:$bound_repo,trace_path:$trace, + created_ts:"2026-07-27T00:00:00Z",operation_id:"op-20260727T000000Z-aaaaaa" + }' > "$project_dir/runs.jsonl" + assurance_sha="$(sha256sum "$sidecar" | awk '{print $1}')" + jq -n --arg assurance_sha "$assurance_sha" --slurpfile a "$sidecar" ' + $a[0] as $sidecar | { + kind:"gate_assurance_attestation_v1", + schema_version:1, + result_sha256:$sidecar.bindings.result_sha256, + assurance_sha256:$assurance_sha, + repo_root:$sidecar.bindings.repo_root, + repo_identity:$sidecar.bindings.repo_identity, + base_commit:$sidecar.bindings.base_commit, + head_commit:$sidecar.bindings.head_commit, + subject_fingerprint:$sidecar.bindings.subject_fingerprint, + run_ids:[$sidecar.dispatch.outcomes[].run_id] + } + ' > "$attestation" +} + +_gate_verify_result_path() { + local slug="$1" run_root + run_root="$( + PM_DISPATCH_STATE_ROOT="$_GATE_VERIFY_STATE_ROOT" \ + _SW_REPO_ROOT="$_GATE_VERIFY_REPO" \ + sw_project_run_dir "gate-$slug" + )" + printf '%s/.gate-results/result.md\n' "$run_root" +} + +_run_canonical_gate_verify() { + local result="$1" + ( + cd "$_GATE_VERIFY_REPO" + PM_DISPATCH_STATE_ROOT="$_GATE_VERIFY_STATE_ROOT" \ + "$PMCTL" gate verify "$result" + ) +} + +_mk_gate_result_v2_legacy_assurance() { + local path="$1" sidecar="${1}.assurance.json" + _mk_gate_result_v2 "$path" + jq ' + .kind = "gate_assurance_v1" | + .schema_version = 1 | + del(.bindings) | + .coordinates.independence = { + implementation_context_isolated:true, + reviewer_topology:"combined-session", + per_reviewer_independent:false, + evidence_status:"verified" + } | + .dispatch.outcomes = [{ + role:"combined",reviewer:null,status:"passed", + run_id:"run-20260727T000000Z-aaaaaa",evidence_status:"verified" + }] | + .provenance = {producer:"pr-gate.sh",policy_source:"canonical"} + ' "$sidecar" > "${sidecar}.tmp" + mv "${sidecar}.tmp" "$sidecar" +} + # ---- 6: gate verify accepts a structurally valid result ---------------------- case_verify_valid() { local name="gate/verify: valid result exits 0" @@ -402,7 +535,297 @@ case_verify_valid() { _mk_gate_result "$result" GO local out code set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e - if [[ "$code" -eq 0 ]] && [[ "$out" == *"gate result OK"* ]]; then + if [[ "$code" -eq 0 ]] && [[ "$out" == *"gate result OK"* ]] \ + && [[ "$out" == *"assurance: unavailable"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_assurance() { + local name="gate/verify: v2 machine assurance exits 0" + should_run "$name" || return 0 + local result="$tmp_root/v2-assurance/result.md" out code + _mk_gate_result_v2 "$result" + set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 0 && "$out" == *"assurance: verified"* \ + && "$out" == *"assurance file:"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_canonical_authorization() { + local name="gate/verify: v2 protected attestation and canonical runs exit 0" + should_run "$name" || return 0 + local result + local out code + result="$(_gate_verify_result_path auth)" + _mk_gate_result_v2_verified "$result" "$_GATE_VERIFY_REPO" + set +e; out="$(_run_canonical_gate_verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 0 && "$out" == *"assurance: verified"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_forged_state_tree_rejected() { + local name="gate/verify: self-consistent noncanonical state tree exits 1" + should_run "$name" || return 0 + local result="$tmp_root/forged/projects/key/runs/gate-forged/.gate-results/result.md" + local out code + _mk_gate_result_v2_verified "$result" "$_GATE_VERIFY_REPO" + set +e; out="$(_run_canonical_gate_verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 \ + && "$out" == *"outside the invoking repository canonical state partition"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_repo_binding_rejected() { + local name="gate/verify: canonical state with wrong repository binding exits 1" + should_run "$name" || return 0 + local result out code + result="$(_gate_verify_result_path repo-binding)" + _mk_gate_result_v2_verified "$result" "/tmp/not-the-invoking-repo" + set +e; out="$(_run_canonical_gate_verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 \ + && "$out" == *"repository binding does not match the invoking repository"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_legacy_assurance_is_unavailable() { + local name="gate/verify: unbound v1 envelope remains readable but unavailable" + should_run "$name" || return 0 + local result="$tmp_root/v2-legacy-envelope/result.md" out code + _mk_gate_result_v2_legacy_assurance "$result" + set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 0 && "$out" == *"assurance: unavailable"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_claim_mismatch() { + local name="gate/verify: v2 coverage partition mismatch exits 1" + should_run "$name" || return 0 + local result="$tmp_root/v2-mismatch/result.md" out code + _mk_gate_result_v2 "$result" + jq '.coordinates.coverage.skipped = []' "${result}.assurance.json" \ + > "${result}.assurance.tmp" + mv "${result}.assurance.tmp" "${result}.assurance.json" + set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 && "$out" == *"structural/claim verification"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_surplus_topology_record() { + local name="gate/verify: v2 surplus topology record exits 1" + should_run "$name" || return 0 + local result="$tmp_root/v2-surplus/result.md" out code + _mk_gate_result_v2 "$result" + jq '.dispatch.outcomes += [{ + role:"synthesis",reviewer:null,status:"passed", + run_id:null,evidence_status:"unavailable" + }]' "${result}.assurance.json" > "${result}.assurance.tmp" + mv "${result}.assurance.tmp" "${result}.assurance.json" + set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 && "$out" == *"structural/claim verification"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_unknown_fields_rejected() { + local name="gate/verify: v2 unknown top-level and nested fields exit 1" + should_run "$name" || return 0 + local variant result out code + for variant in top-level nested; do + result="$tmp_root/v2-unknown-$variant/result.md" + _mk_gate_result_v2 "$result" + if [[ "$variant" == top-level ]]; then + jq '.unexpected = true' "${result}.assurance.json" > "${result}.assurance.tmp" + else + jq '.coordinates.tier.unexpected = true' "${result}.assurance.json" \ + > "${result}.assurance.tmp" + fi + mv "${result}.assurance.tmp" "${result}.assurance.json" + set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -ne 1 || "$out" != *"structural/claim verification"* ]]; then + fail "$name" "$variant code=$code out=$out" + return + fi + done + pass "$name" +} + +case_verify_v2_result_binding_tamper() { + local name="gate/verify: v2 changed result digest exits 1" + should_run "$name" || return 0 + local result="$tmp_root/v2-result-tamper/projects/key/runs/gate-test/.gate-results/result.md" + local out code + _mk_gate_result_v2_verified "$result" + printf '\npost-finalization mutation\n' >> "$result" + set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 && "$out" == *"structural/claim verification"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_sidecar_attestation_tamper() { + local name="gate/verify: v2 substituted sidecar exits 1" + should_run "$name" || return 0 + local result + local out code sidecar + result="$(_gate_verify_result_path sidecar-tamper)" + sidecar="${result}.assurance.json" + _mk_gate_result_v2_verified "$result" "$_GATE_VERIFY_REPO" + jq '.coordinates.tier.evidence_floor = "forged"' "$sidecar" > "${sidecar}.tmp" + mv "${sidecar}.tmp" "$sidecar" + set +e; out="$(_run_canonical_gate_verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 && "$out" == *"protected attestation mismatch"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_subject_binding_tamper() { + local name="gate/verify: v2 changed subject fingerprint exits 1" + should_run "$name" || return 0 + local result + local out code sidecar + result="$(_gate_verify_result_path subject-tamper)" + sidecar="${result}.assurance.json" + _mk_gate_result_v2_verified "$result" "$_GATE_VERIFY_REPO" + jq '.bindings.subject_fingerprint = ("f" * 64)' "$sidecar" > "${sidecar}.tmp" + mv "${sidecar}.tmp" "$sidecar" + set +e; out="$(_run_canonical_gate_verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 && "$out" == *"protected attestation mismatch"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_canonical_run_mismatch() { + local name="gate/verify: v2 unresolvable canonical run exits 1" + should_run "$name" || return 0 + local result + local out code run_root project_dir + result="$(_gate_verify_result_path run-tamper)" + _mk_gate_result_v2_verified "$result" "$_GATE_VERIFY_REPO" + run_root="$(dirname "$(dirname "$result")")" + project_dir="$(dirname "$(dirname "$run_root")")" + jq '.state = "failed" | .exit_code = 1' "$project_dir/runs.jsonl" \ + > "$project_dir/runs.tmp" + mv "$project_dir/runs.tmp" "$project_dir/runs.jsonl" + set +e; out="$(_run_canonical_gate_verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 && "$out" == *"canonical run records"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_publication_race_retries() { + local name="gate/verify: canonical v2 publication race retries" + should_run "$name" || return 0 + local result sidecar staged out code publisher + result="$(_gate_verify_result_path publication-race)" + sidecar="${result}.assurance.json" + staged="${sidecar}.staged" + _mk_gate_result_v2 "$result" + mv "$sidecar" "$staged" + ( + sleep 0.2 + mv "$staged" "$sidecar" + ) & + publisher=$! + set +e; out="$(_run_canonical_gate_verify "$result" 2>&1)"; code=$?; set -e + wait "$publisher" + if [[ "$code" -eq 0 && "$out" == *"gate result OK"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_attestation_publication_race_retries() { + local name="gate/verify: canonical v2 attestation publication race retries" + should_run "$name" || return 0 + local result run_root attestation staged out code publisher + result="$(_gate_verify_result_path attestation-race)" + _mk_gate_result_v2_verified "$result" "$_GATE_VERIFY_REPO" + run_root="$(dirname "$(dirname "$result")")" + attestation="$run_root/gate-assurance-20260727-000000.attestation.json" + staged="${attestation}.staged" + mv "$attestation" "$staged" + ( + sleep 0.2 + mv "$staged" "$attestation" + ) & + publisher=$! + set +e; out="$(_run_canonical_gate_verify "$result" 2>&1)"; code=$?; set -e + wait "$publisher" + if [[ "$code" -eq 0 && "$out" == *"assurance: verified"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_pointer_escape() { + local name="gate/verify: v2 sidecar pointer escape exits 1" + should_run "$name" || return 0 + local result="$tmp_root/v2-pointer/result.md" out code + _mk_gate_result_v2 "$result" + sed -i 's|^gate_assurance:.*|gate_assurance: ../outside.json|' "$result" + set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 && "$out" == *"bounded sibling"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_missing_sidecar() { + local name="gate/verify: v2 missing assurance sidecar exits 1" + should_run "$name" || return 0 + local result="$tmp_root/v2-missing-sidecar/result.md" out code + _mk_gate_result_v2 "$result" + rm -f "${result}.assurance.json" + set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 && "$out" == *"sidecar missing or empty"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + +case_verify_v2_empty_sidecar() { + local name="gate/verify: v2 empty assurance sidecar exits 1" + should_run "$name" || return 0 + local result="$tmp_root/v2-empty-sidecar/result.md" out code + _mk_gate_result_v2 "$result" + : > "${result}.assurance.json" + set +e; out="$("$PMCTL" gate verify "$result" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 1 && "$out" == *"sidecar missing or empty"* ]]; then pass "$name" else fail "$name" "code=$code out=$out" @@ -874,6 +1297,23 @@ case_cd_missing_value_rejected case_pmctl_routing case_help_bypasses_detached_default case_verify_valid +case_verify_v2_assurance +case_verify_v2_canonical_authorization +case_verify_v2_forged_state_tree_rejected +case_verify_v2_repo_binding_rejected +case_verify_v2_legacy_assurance_is_unavailable +case_verify_v2_claim_mismatch +case_verify_v2_surplus_topology_record +case_verify_v2_unknown_fields_rejected +case_verify_v2_result_binding_tamper +case_verify_v2_sidecar_attestation_tamper +case_verify_v2_subject_binding_tamper +case_verify_v2_canonical_run_mismatch +case_verify_v2_publication_race_retries +case_verify_v2_attestation_publication_race_retries +case_verify_v2_pointer_escape +case_verify_v2_missing_sidecar +case_verify_v2_empty_sidecar case_verify_empty case_verify_no_final case_verify_parity_mismatch diff --git a/tests/shell/test-pr-gate-profile.sh b/tests/shell/test-pr-gate-profile.sh index 701c893b..b8859790 100755 --- a/tests/shell/test-pr-gate-profile.sh +++ b/tests/shell/test-pr-gate-profile.sh @@ -43,7 +43,7 @@ create_runner() { mkdir -p "$dir/lib" cp -R "$REPO_ROOT/runtime/lib/." "$dir/lib/" local cmd - for cmd in bash git date readlink dirname basename cp mkdir touch ln cat grep sort wc awk sed mktemp rm head tail tr true false sha256sum shasum; do + for cmd in bash git date readlink dirname basename cp mkdir touch ln cat grep sort wc awk sed mktemp rm mv find head tail tr true false sha256sum shasum jq; do src="$(command -v "$cmd" 2>/dev/null || true)" if [[ -n "$src" ]]; then ln -sf "$src" "$dir/$cmd" @@ -88,7 +88,7 @@ fi if [[ -n "$output_path" ]]; then mkdir -p "$(dirname "$output_path")" if [[ "$brief_file" == *-synthesis.md ]]; then - printf -- '---\ngate_result_version: pr_gate_result_v1\nfinal: GO\ntier: standard\nmode: sequential\nmost_severe: advise\nreviewers:\n critic: advise\nescalation:\n recommended: false\n reviewers: []\n reason: []\n---\n\n# PR-Gate Result — stub tier\n**Date**: 2026-05-17\n**Reviewers**: stub\n**Not reviewed**: none\n\n## cross-check\nnone\n\n## Gate Conclusion\n**Overall verdict**: advise\n**Most severe individual verdict**: advise\nFinal: GO\n' > "$output_path" + printf -- '---\ngate_result_version: pr_gate_result_v1\nfinal: GO\ntier: standard\nmode: parallel\nmost_severe: advise\nreviewers:\n critic: advise\nescalation:\n recommended: false\n reviewers: []\n reason: []\n---\n\n# PR-Gate Result — stub tier\n**Date**: 2026-05-17\n**Reviewers**: stub\n**Not reviewed**: none\n\n## cross-check\nnone\n\n## Gate Conclusion\n**Overall verdict**: advise\n**Most severe individual verdict**: advise\nFinal: GO\n' > "$output_path" else printf -- '---\ngate_result_version: pr_gate_result_v1\nfinal: GO\ntier: standard\nmode: sequential\nmost_severe: advise\nreviewers:\n critic: advise\nescalation:\n recommended: false\n reviewers: []\n reason: []\n---\n\n## stub-reviewer — advise\nVerdict: advise. Stub output.\nFinal: GO\n' > "$output_path" fi @@ -139,7 +139,7 @@ fi if [[ -n "$output_path" ]]; then mkdir -p "$(dirname "$output_path")" if [[ "$brief_file" == *-synthesis.md ]]; then - printf -- '---\ngate_result_version: pr_gate_result_v1\nfinal: GO\ntier: standard\nmode: sequential\nmost_severe: advise\nreviewers:\n critic: advise\nescalation:\n recommended: false\n reviewers: []\n reason: []\n---\n\n# PR-Gate Result — stub tier\n**Date**: 2026-05-17\n**Reviewers**: stub\n**Not reviewed**: none\n\n## cross-check\nnone\n\n## Gate Conclusion\n**Overall verdict**: advise\n**Most severe individual verdict**: advise\nFinal: GO\n' > "$output_path" + printf -- '---\ngate_result_version: pr_gate_result_v1\nfinal: GO\ntier: standard\nmode: parallel\nmost_severe: advise\nreviewers:\n critic: advise\nescalation:\n recommended: false\n reviewers: []\n reason: []\n---\n\n# PR-Gate Result — stub tier\n**Date**: 2026-05-17\n**Reviewers**: stub\n**Not reviewed**: none\n\n## cross-check\nnone\n\n## Gate Conclusion\n**Overall verdict**: advise\n**Most severe individual verdict**: advise\nFinal: GO\n' > "$output_path" else printf -- '---\ngate_result_version: pr_gate_result_v1\nfinal: GO\ntier: standard\nmode: sequential\nmost_severe: advise\nreviewers:\n critic: advise\nescalation:\n recommended: false\n reviewers: []\n reason: []\n---\n\n## stub-reviewer — advise\nVerdict: advise. Stub output.\nFinal: GO\n' > "$output_path" fi @@ -184,7 +184,7 @@ build_no_codex_path() { local dir="$1/no-codex-bin" mkdir -p "$dir" local cmd - for cmd in bash git date readlink dirname basename cp mkdir touch ln cat grep sort wc awk sed mktemp rm cat sort head tail tr wc awk date sha256sum shasum git; do + for cmd in bash git date readlink dirname basename cp mkdir touch ln cat grep sort wc awk sed mktemp rm mv find cat sort head tail tr wc awk date sha256sum shasum git jq; do src="$(command -v "$cmd" 2>/dev/null || true)" if [[ -n "$src" ]]; then ln -sf "$src" "$dir/$cmd" diff --git a/tests/shell/test-pr-gate.sh b/tests/shell/test-pr-gate.sh index 811e0cc8..962575cf 100755 --- a/tests/shell/test-pr-gate.sh +++ b/tests/shell/test-pr-gate.sh @@ -194,6 +194,11 @@ write_frontmatter_stub_gate_result() { local output_path="$1" local final_verdict="${2:-GO}" local final_line="Final: ${final_verdict}" + local resolved_tier resolved_mode + resolved_tier="$(awk '/^[[:space:]]*tier\.resolved:/ {print $2; exit}' "$brief_file")" + resolved_mode="$(awk '/^[[:space:]]*mode\.resolved:/ {print $2; exit}' "$brief_file")" + : "${resolved_tier:=express}" + : "${resolved_mode:=parallel}" # Regression seam: when CODEX_GATE_STUB_BOLD_FINAL=1, emit the Final # line wrapped in markdown bold (simulates codex applying prose emphasis). @@ -206,8 +211,8 @@ write_frontmatter_stub_gate_result() { --- gate_result_version: pr_gate_result_v1 final: ${CODEX_GATE_STUB_FRONTMATTER_FINAL:-${final_verdict}} -tier: express -mode: parallel +tier: ${resolved_tier} +mode: ${resolved_mode} most_severe: approve reviewers: critic: approve @@ -1365,6 +1370,20 @@ test_parallel_launches_per_reviewer() { assert_file_contains "$name" "$out" "[parallel] launched critic" || return assert_file_contains "$name" "$out" "[parallel] launched qa-tester" || return assert_file_contains "$name" "$out" "[synthesis] running PM consolidation" || return + local result_path + result_path="$(awk -F'result: ' '/^result: /{path=$2} END{print path}' "$out")" + jq -e ' + .coordinates.mode.resolved == "parallel" and + .coordinates.independence.evidence_status == "unavailable" and + .coordinates.independence.per_reviewer_independent == null and + ([.dispatch.outcomes[] | select(.role == "reviewer") | .reviewer] | sort) == + ["critic","qa-tester"] and + ([.dispatch.outcomes[] | select(.role == "synthesis")] | length) == 1 and + all(.dispatch.outcomes[]; .run_id == null and .evidence_status == "unavailable") + ' "${result_path}.assurance.json" >/dev/null || { + fail "$name" "copy-mode parallel envelope claimed unavailable evidence incorrectly" + return + } pass "$name" } @@ -3183,10 +3202,30 @@ test_gate_result_frontmatter_and_escalation() { fi local frontmatter frontmatter="$(sed -n "1,${frontmatter_end}p" "$result")" - if ! printf '%s\n' "$frontmatter" | grep -q '^gate_result_version: pr_gate_result_v1$'; then - fail "$name" "frontmatter missing gate_result_version: pr_gate_result_v1" + if ! printf '%s\n' "$frontmatter" | grep -q '^gate_result_version: pr_gate_result_v2$'; then + fail "$name" "frontmatter missing gate_result_version: pr_gate_result_v2" + return + fi + if ! printf '%s\n' "$frontmatter" | grep -q '^gate_assurance: result.md.assurance.json$'; then + fail "$name" "frontmatter missing bounded gate_assurance sidecar pointer" return fi + if [[ ! -s "${result}.assurance.json" ]]; then + fail "$name" "machine-owned assurance sidecar missing" + return + fi + jq -e ' + .coordinates.mode.resolved == "sequential" and + .coordinates.independence.evidence_status == "unavailable" and + .coordinates.independence.per_reviewer_independent == null and + .dispatch.outcomes == [{ + role:"combined",reviewer:null,status:"passed",run_id:null, + evidence_status:"unavailable" + }] + ' "${result}.assurance.json" >/dev/null || { + fail "$name" "copy-mode sequential envelope did not degrade truthfully" + return + } if ! printf '%s\n' "$frontmatter" | grep -Eq '^final: (GO|NO-GO)$'; then fail "$name" "missing frontmatter final in GO|NO-GO form" return @@ -3230,6 +3269,103 @@ test_gate_result_frontmatter_and_escalation() { pass "$name" } +# Behavior: repo-layout dispatches record the actual pmctl run id in the +# machine-owned assurance envelope instead of claiming verified independence +# from executor prose. +test_repo_layout_captures_dispatch_run_id() { + local name="gate-assurance/repo-layout-captures-run-id" + should_run "$name" || return 0 + local dir source_runner layout home repo out err result project_key run_dir runs_file + dir="$TMP_ROOT/$name" + source_runner="$dir/source-runner" + layout="$dir/layout" + home="$dir/home" + repo="$dir/repo" + out="$dir/out" + err="$dir/err" + mkdir -p "$dir" "$layout/runtime/bin" "$layout/runtime/lib" "$layout/core/policy" + create_runner "$source_runner" + cp "$source_runner/pr-gate.sh" "$layout/runtime/bin/pr-gate.sh" + cp -R "$source_runner/lib/." "$layout/runtime/lib/" + cp -R "$source_runner/core/policy/." "$layout/core/policy/" + cp -R "$REPO_ROOT/agents" "$layout/agents" + cp -R "$REPO_ROOT/adapters" "$layout/adapters" + cp "$source_runner/adapters/codex/dispatch.sh" "$layout/adapters/codex/dispatch.sh" + chmod +x "$layout/runtime/bin/pr-gate.sh" "$layout/adapters/codex/dispatch.sh" + cat > "$layout/runtime/lib/pmctl-dispatch.sh" <<'STUB_PMCTL' +pmctl_dispatch_run() { + local root="$1" brief="" work="" timeout="" + local capture_mode + shift + while [[ $# -gt 0 ]]; do + case "$1" in + --brief-file) brief="$2"; shift 2 ;; + --cd) work="$2"; shift 2 ;; + --timeout) timeout="$2"; shift 2 ;; + *) shift ;; + esac + done + capture_mode="$(stat -c '%a' "$GATE_ASSURANCE_CAPTURE_DIR" 2>/dev/null)" || return 2 + case "$GATE_ASSURANCE_CAPTURE_DIR" in + /tmp/pm-gate-assurance-*) ;; + *) return 2 ;; + esac + [[ -d "$GATE_ASSURANCE_CAPTURE_DIR" && ! -L "$GATE_ASSURANCE_CAPTURE_DIR" \ + && "$capture_mode" == 700 ]] || return 2 + "$root/adapters/codex/dispatch.sh" --brief-file "$brief" --cd "$work" --timeout "$timeout" + mkdir -p "$PM_DISPATCH_TRACE_DIR" + printf 'trace\n' > "$PM_DISPATCH_TRACE_DIR/test.last" + jq -nc --arg work "$work" --arg trace "$PM_DISPATCH_TRACE_DIR/test.last" '{ + schema_version:3,id:"run-20260727T000000Z-aaaaaa",task_id:"UNKN-0", + executor:"codex",state:"ok",exit_code:0,model:"default", + brief_file:"/tmp/brief.md",working_dir:$work,trace_path:$trace, + created_ts:"2026-07-27T00:00:00Z",operation_id:"op-20260727T000000Z-aaaaaa" + }' >> "$CODEX_GATE_TEST_RUNS_FILE" + printf 'run-20260727T000000Z-aaaaaa\n' +} +pmctl_dispatch_wait() { return 0; } +STUB_PMCTL + create_agents "$home" critic qa-tester architecture-reviewer security-reviewer risk-reviewer + create_repo "$repo" docs + project_key="$(printf '%s\n' "$repo" | sha1sum | awk '{print $1}')" + run_dir="$dir/state/projects/$project_key/runs/gate-fixture" + runs_file="$dir/state/projects/$project_key/runs.jsonl" + mkdir -p "$run_dir" "$(dirname "$runs_file")" + + local code=0 + set +e + HOME="$home" PM_DISPATCH_STATE_ROOT="$dir/state" CODEX_GATE_TEST_RUNS_FILE="$runs_file" \ + "$layout/runtime/bin/pr-gate.sh" --cd "$repo" --base main --executor codex \ + --mode sequential --run-dir "$run_dir" > "$out" 2> "$err" + code=$? + set -e + [[ "$code" -eq 0 ]] || { + fail "$name" "exit $code, expected 0: $(tail -n 20 "$err" 2>/dev/null)" + return + } + result="$(awk -F'result: ' '/^result: /{path=$2} END{print path}' "$out")" + jq -e ' + .coordinates.independence.evidence_status == "verified" and + .coordinates.independence.per_reviewer_independent == false and + .dispatch.outcomes == [{ + role:"combined",reviewer:null,status:"passed", + run_id:"run-20260727T000000Z-aaaaaa",evidence_status:"verified" + }] + ' "${result}.assurance.json" >/dev/null || { + fail "$name" "assurance envelope did not capture the repo-layout run id" + return + } + if ! ( + cd "$repo" + PM_DISPATCH_STATE_ROOT="$dir/state" \ + "$REPO_ROOT/cli/pmctl" gate verify "$result" >/dev/null 2>&1 + ); then + fail "$name" "repo-layout assurance did not validate against protected canonical evidence" + return + fi + pass "$name" +} + # Behavior: the parallel gate result body still carries exactly one # plain-text Final: (GO|NO-GO) line, preserving the pre-frontmatter # back-compat contract that downstream consumers grep for. @@ -3662,6 +3798,7 @@ run_test test_sequential_combined_brief_validates run_test test_parallel_reviewer_brief_validates run_test test_parallel_synthesis_brief_validates run_test test_gate_result_frontmatter_and_escalation +run_test test_repo_layout_captures_dispatch_run_id run_test test_gate_result_final_line_back_compat run_test test_frontmatter_escalation_parity run_test test_failed_reviewer_aborts_gate @@ -3875,6 +4012,9 @@ test_post_gate_hook_runs() { mkdir -p "$repo/.pm-dispatch" { printf '#!/usr/bin/env bash\n' + printf 'result="$(find .gate-results -maxdepth 1 -type f -name '\''gate-*.md'\'' | head -n 1)"\n' + printf 'grep -q '\''^gate_result_version: pr_gate_result_v1$'\'' "$result" || exit 8\n' + printf 'test ! -e "${result}.assurance.json" || exit 9\n' printf 'touch "%s"\n' "$hook_marker" } > "$repo/.pm-dispatch/post-gate.sh" chmod +x "$repo/.pm-dispatch/post-gate.sh" @@ -3891,6 +4031,13 @@ test_post_gate_hook_runs() { fail "$name" "post-gate hook did not run (marker missing)" return fi + local result_path + result_path="$(awk -F'result: ' '/^result: /{path=$2} END{print path}' "$out")" + assert_file_contains "$name" "$result_path" "gate_result_version: pr_gate_result_v2" || return + if [[ ! -s "${result_path}.assurance.json" ]]; then + fail "$name" "assurance was not finalized after the successful post-gate hook" + return + fi pass "$name" } @@ -4370,6 +4517,18 @@ test_targeted_pass_references_initial_result() { fail "$name" "--targeted critic did not scope reviewers — qa-tester was launched" return fi + local result_path + result_path="$(awk -F'result: ' '/^result: /{path=$2} END{print path}' "$out")" + jq -e --arg initial "$initial" ' + .coordinates.pass.resolved == "targeted" and + .coordinates.pass.scope == "remediation-delta" and + .coordinates.pass.initial_result == $initial and + .coordinates.coverage.requested == ["critic"] and + .coordinates.coverage.selected == ["critic"] + ' "${result_path}.assurance.json" >/dev/null || { + fail "$name" "targeted assurance envelope lost its initial reference or coverage" + return + } pass "$name" } @@ -4433,6 +4592,90 @@ test_targeted_output_cannot_overwrite_initial_result() { pass "$name" } +# Behavior: the deterministic v2 sidecar path cannot overwrite a targeted +# pass's referenced initial result. +test_targeted_sidecar_cannot_overwrite_initial_result() { + local name="targeted-sidecar-cannot-overwrite-initial-result" + should_run "$name" || return 0 + local dir="$TMP_ROOT/$name" + local home="$dir/home" repo="$dir/repo" runner="$dir/runner" + local out="$dir/out" err="$dir/err" + local result="$dir/result.md" initial + initial="${result}.assurance.json" + mkdir -p "$dir" + create_runner "$runner" + create_agents "$home" critic qa-tester architecture-reviewer security-reviewer risk-reviewer + create_repo "$repo" docs + write_valid_initial_gate_result "$initial" + + local code=0 + set +e + run_gate "$home" "$runner" "$repo" "$out" "$err" --base main \ + --targeted critic --initial-result "$initial" --output "$result" + code=$? + set -e + [[ "$code" -eq 2 ]] || { + fail "$name" "sidecar/initial collision exited $code, expected 2" + return + } + assert_file_contains "$name" "$err" "assurance sidecar must not overwrite" || return + assert_not_contains "$name" "$out" "DISPATCH_STUB" || return + pass "$name" +} + +# Behavior: a pre-existing symlink, non-regular file, or hardlink at the +# deterministic assurance destination is rejected before reviewer dispatch. +# Steps: prepare each unsafe destination type, run with an explicit output, and +# assert exit 2, no dispatch, and no write through linked targets. +test_assurance_unsafe_destinations_rejected() { + local name="assurance-unsafe-destinations-rejected" + should_run "$name" || return 0 + local dir="$TMP_ROOT/$name" + local home="$dir/home" repo="$dir/repo" runner="$dir/runner" + local kind result sidecar target out err code + mkdir -p "$dir" + create_runner "$runner" + create_agents "$home" critic qa-tester architecture-reviewer security-reviewer risk-reviewer + create_repo "$repo" docs + + for kind in symlink directory hardlink; do + result="$dir/result-$kind.md" + sidecar="${result}.assurance.json" + target="$dir/target-$kind" + out="$dir/out-$kind" + err="$dir/err-$kind" + case "$kind" in + symlink) + printf 'sentinel\n' > "$target" + ln -s "$target" "$sidecar" + ;; + directory) + mkdir -p "$sidecar" + ;; + hardlink) + printf 'sentinel\n' > "$target" + ln "$target" "$sidecar" + ;; + esac + + code=0 + set +e + run_gate "$home" "$runner" "$repo" "$out" "$err" --base main --output "$result" + code=$? + set -e + if [[ "$code" -ne 2 ]]; then + fail "$name" "$kind destination exited $code, expected 2" + return + fi + assert_not_contains "$name" "$out" "DISPATCH_STUB" || return + if [[ "$kind" != directory && "$(<"$target")" != "sentinel" ]]; then + fail "$name" "$kind destination modified its linked target" + return + fi + done + pass "$name" +} + # Behavior: canonical and compatibility mode spellings fail closed when they # request different topologies. # Steps: combine --mode parallel with --sequential and assert a controlled @@ -5005,7 +5248,7 @@ _cc469_build_pmctl_less_path() { local minpath="$runner/.no-pmctl-bin" mkdir -p "$minpath" local cmd - for cmd in bash git date readlink dirname basename cp mkdir touch ln cat grep sort wc awk sed mktemp rm head tail tr true false sha256sum shasum; do + for cmd in bash git date readlink dirname basename cp mv mkdir touch ln cat grep sort wc awk sed mktemp rm head tail tr true false sha256sum shasum find jq; do local src src="$(command -v "$cmd" 2>/dev/null || true)" [[ -n "$src" ]] && ln -sf "$src" "$minpath/$cmd" @@ -5016,6 +5259,40 @@ _cc469_build_pmctl_less_path() { REPLY="$minpath:$_codex_stub_bin" } +# Behavior: the mandatory jq dependency is checked before any reviewer work. +# Steps: build the minimal standalone PATH, remove jq, and assert a clear +# configuration error with no dispatch. +test_missing_jq_fails_before_dispatch() { + local name="missing-jq-fails-before-dispatch" + should_run "$name" || return 0 + local dir="$TMP_ROOT/$name" + local home="$dir/home" repo="$dir/repo" runner="$dir/runner" + local out="$dir/out" err="$dir/err" + mkdir -p "$dir" + create_runner "$runner" + create_agents "$home" critic qa-tester architecture-reviewer security-reviewer risk-reviewer + create_repo "$repo" docs + + local REPLY + _cc469_build_pmctl_less_path "$runner" + local minpath="$REPLY" + rm -f "$runner/.no-pmctl-bin/jq" + + local code=0 + set +e + HOME="$home" PATH="$minpath" \ + "$runner/pr-gate.sh" --cd "$repo" --base main > "$out" 2> "$err" + code=$? + set -e + if [[ "$code" -ne 2 ]]; then + fail "$name" "exit $code, expected 2" + return + fi + assert_file_contains "$name" "$err" "requires jq on PATH" || return + assert_not_contains "$name" "$out" "DISPATCH_STUB" || return + pass "$name" +} + # Behavior: when the real `pmctl` is not resolvable on PATH, the sequential # combined reviewer brief's guard-check instruction falls back to the # absolute path of the sibling cli/pmctl next to pr-gate.sh, instead of the @@ -5145,6 +5422,8 @@ run_test test_unknown_arg_message run_test test_targeted_pass_references_initial_result run_test test_targeted_requires_initial_result run_test test_targeted_output_cannot_overwrite_initial_result +run_test test_targeted_sidecar_cannot_overwrite_initial_result +run_test test_assurance_unsafe_destinations_rejected run_test test_conflicting_mode_options_are_rejected run_test test_equivalent_mode_spellings_are_accepted run_test test_invalid_assurance_inputs_are_rejected @@ -5375,28 +5654,45 @@ test_relative_output_normalized_to_absolute() { pass "$name" } -# Behavior: pr-gate.sh's inline copy of gate_result_verify (for copy-mode, -# run standalone without runtime/lib/) stays identical (modulo -# indentation) to runtime/lib/gate-result-verify.sh -- a drifted copy -# would silently diverge the gate's integrity contract. -# Steps: extract the gate_result_verify function body from both the lib -# and the pr-gate.sh inline fallback, and assert the two bodies match -# exactly. +# Behavior: the generated standalone copy-mode verifier fallback is current. +# Steps: run the checked-in generator in check mode, then retain function-level +# diagnostics if a future generator bug produces an incomplete block. test_inline_fallback_matches_lib() { local name="inline-fallback-matches-lib" should_run "$name" || return 0 - local lib_body inline_body - lib_body="$(awk '/^gate_result_verify\(\) \{/{f=1} f{print} f&&/^\}$/{exit}' "$REPO_ROOT/runtime/lib/gate-result-verify.sh" | sed 's/^[[:space:]]*//')" - inline_body="$(awk '/gate_result_verify\(\) \{/{f=1} f{print} f&&/^ \}$/{exit}' "$REPO_ROOT/runtime/bin/pr-gate.sh" | sed 's/^[[:space:]]*//')" - if [[ -z "$lib_body" || -z "$inline_body" ]]; then - fail "$name" "could not extract gate_result_verify from lib and/or pr-gate.sh inline fallback" + local function_name lib_body inline_body + if ! bash "$REPO_ROOT/tools/generate-gate-result-verifier-fallback.sh" --check; then + fail "$name" "generated verifier fallback is stale" return fi - if [[ "$lib_body" == "$inline_body" ]]; then - pass "$name" - else - fail "$name" "inline gate_result_verify in pr-gate.sh drifted from runtime/lib/gate-result-verify.sh -- keep them in sync" - fi + local -a verifier_functions=( + gate_result_verdict_verify + _gate_result_frontmatter_value + _gate_result_sha256_file + gate_assurance_verify + gate_result_verify + ) + for function_name in "${verifier_functions[@]}"; do + lib_body="$(awk -v signature="$function_name() {" ' + $0 == signature {found=1} + found {print} + found && $0 == "}" {exit} + ' "$REPO_ROOT/runtime/lib/gate-result-verify.sh" | sed 's/^[[:space:]]*//')" + inline_body="$(awk -v signature=" $function_name() {" ' + $0 == signature {found=1} + found {print} + found && $0 == " }" {exit} + ' "$REPO_ROOT/runtime/bin/pr-gate.sh" | sed 's/^[[:space:]]*//')" + if [[ -z "$lib_body" || -z "$inline_body" ]]; then + fail "$name" "could not extract $function_name from shared and/or inline verifier" + return + fi + if [[ "$lib_body" != "$inline_body" ]]; then + fail "$name" "inline $function_name drifted from runtime/lib/gate-result-verify.sh" + return + fi + done + pass "$name" } # Behavior: a repo-root .gate-overrides.md is injected into the sequential @@ -5621,6 +5917,7 @@ run_test test_parallel_reviewer_brief_has_guard_constraint run_test test_seq_brief_guard_absolute_path_when_pmctl_not_on_path run_test test_parallel_reviewer_brief_guard_absolute_path_when_pmctl_not_on_path run_test test_claude_seq_brief_guard_stays_bare_pmctl_when_pmctl_not_on_path +run_test test_missing_jq_fails_before_dispatch run_test test_relative_output_normalized_to_absolute run_test test_inline_fallback_matches_lib run_test test_brief_major_suggests_full @@ -5965,6 +6262,17 @@ test_gate_artifacts_land_out_of_repo() { return fi assert_file_contains "$name" "$result_path" "test_evidence: $evidence_path" || return + local assurance_pointer assurance_path + assurance_pointer="$(awk '$1 == "gate_assurance:" {print $2; exit}' "$result_path")" + assurance_path="$(dirname "$result_path")/$assurance_pointer" + if [[ ! -s "$assurance_path" ]]; then + fail "$name" "gate assurance sidecar was not relocated with the result" + return + fi + if ! "$REPO_ROOT/cli/pmctl" gate verify "$result_path" >/dev/null 2>&1; then + fail "$name" "relocated result/assurance pointer failed shared verification" + return + fi # repo must NOT have a .gate-results dir (--run-dir should have redirected it). if [[ -d "$repo/.gate-results" ]]; then fail "$name" ".gate-results appeared inside repo -- --run-dir did not redirect results" diff --git a/tools/generate-gate-result-verifier-fallback.sh b/tools/generate-gate-result-verifier-fallback.sh new file mode 100755 index 00000000..5dd552c7 --- /dev/null +++ b/tools/generate-gate-result-verifier-fallback.sh @@ -0,0 +1,83 @@ +#!/usr/bin/env bash +# Generate pr-gate.sh's standalone verifier fallback from the shared library. + +set -euo pipefail + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +source_file="$repo_root/runtime/lib/gate-result-verify.sh" +target_file="$repo_root/runtime/bin/pr-gate.sh" +mode="${1:-sync}" + +case "$mode" in + sync | --check) ;; + *) + printf 'usage: %s [sync|--check]\n' "$0" >&2 + exit 2 + ;; +esac + +block_file="$(mktemp)" +generated_file="$(mktemp)" +cleanup() { + rm -f -- "$block_file" "$generated_file" +} +trap cleanup EXIT + +functions=( + gate_result_verdict_verify + _gate_result_frontmatter_value + _gate_result_sha256_file + gate_assurance_verify + gate_result_verify +) + +for function_name in "${functions[@]}"; do + awk -v signature="$function_name() {" ' + $0 == signature { found = 1 } + found { + if ($0 == "") print "" + else print " " $0 + } + found && $0 == "}" { exit } + ' "$source_file" >> "$block_file" + printf '\n' >> "$block_file" +done + +awk -v block_file="$block_file" ' + BEGIN { + start = " # gate-result-verifier-fallback:start" + finish = " # gate-result-verifier-fallback:end" + } + $0 == start { + print + while ((getline generated < block_file) > 0) print generated + close(block_file) + replacing = 1 + next + } + replacing && $0 == finish { + replacing = 0 + print + next + } + !replacing { print } + END { + if (replacing) { + print "generator: unterminated fallback marker" > "/dev/stderr" + exit 1 + } + } +' "$target_file" > "$generated_file" + +if [[ "$mode" == --check ]]; then + if ! cmp -s "$target_file" "$generated_file"; then + printf 'generated verifier fallback is stale; run: %s sync\n' "$0" >&2 + exit 1 + fi + exit 0 +fi + +if ! cmp -s "$target_file" "$generated_file"; then + chmod --reference="$target_file" "$generated_file" 2>/dev/null || true + mv -- "$generated_file" "$target_file" +fi