From 5f4562d54ea67a0f208aa44126f7352260617a2b Mon Sep 17 00:00:00 2001 From: Stefan Ayala Date: Mon, 10 Aug 2026 04:27:00 -0700 Subject: [PATCH] fix: make native review proof-aware --- .agents/skills/sdlc/SKILL.md | 3 ++- PROVE-IT.md | 13 +++++++++++-- README.md | 15 +++++++++++++-- RELEASE.md | 6 ++++-- SDLC-LOOP.md | 3 ++- lib/codex-config.sh | 2 +- lib/refresh-manifest-hashes.cjs | 3 --- setup.sh | 2 +- skill-sources/sdlc/SKILL.template.md | 2 +- templates/AGENTS.baseline.md | 3 ++- templates/AGENTS.md.tmpl | 3 ++- tests/test-adapter.sh | 4 ++++ tests/test-packaging.sh | 19 ++++++++++++++++--- tests/test-release.sh | 22 ++++++++++++++++++++-- tests/test-skill.sh | 23 +++++++++++++++++++++++ tests/test-update.sh | 7 +++++-- update.sh | 4 ++++ 17 files changed, 111 insertions(+), 23 deletions(-) diff --git a/.agents/skills/sdlc/SKILL.md b/.agents/skills/sdlc/SKILL.md index bcccae3..c31542c 100644 --- a/.agents/skills/sdlc/SKILL.md +++ b/.agents/skills/sdlc/SKILL.md @@ -28,7 +28,8 @@ Use this skill for implementation, bug-fix, refactor, testing, release, publish, 10. Review is mandatory. The portable contract is review behavior, not a slash-command name. Use native Codex review when appropriate: `codex review --uncommitted` before commit, `codex review --base ` for branch or PR-sized diffs, and `codex review --commit ` for a specific commit. Use `codex -c 'model_reasoning_effort="high"' review --uncommitted` for an enforced Sol-high gate, especially from `mixed`; apply the same prefix to `--base` or `--commit` reviews. - Pass the fresh proof command and result in the custom review prompt and say `Do not rerun tests`; the reviewer inspects the diff and evidence. Missing or stale proof is a blocker to report, not permission to launch another broad suite. + Run one broad proof run total on the frozen candidate through the proof-stamping entrypoint. In this repo, use `node .codex/hooks/git-guard.cjs prove --reviewed --check "node scripts/run-proof-suite.cjs"`; do not run the suite directly and then rerun it through the guard. + Use a prompt-only review when supplying custom proof-aware instructions. A custom prompt must not be combined with `--uncommitted`, `--base`, or `--commit`; those predefined target flags are for reviews without a custom prompt. Include the exact base identity, frozen candidate tree identity, fresh proof command, and result, and say `Do not rerun tests`. Targeted verification is allowed only for a concrete suspected defect; never rerun the broad suite. Missing or stale proof is a blocker to report, not permission to launch another broad suite. Reviewer role: inspect the frozen diff and return prioritized code-review findings only; do not edit, implement, run tests, re-plan, or perform follow-up work. The builder owns every correction through the normal SDLC loop. `review_model` controls native Codex review model selection but does not set review reasoning independently. `auto_review` is for eligible approval prompts, not code-diff review. Do not require `/autoreview` unless the current Codex host exposes it as a verified feature. At each coherent green slice, author-review the exact incremental diff before committing. Once the cumulative candidate is stable, freeze it, run one fresh broad proof, and review the full base-to-candidate diff once. A relevant correction invalidates that completion proof; use narrow delta checks while fixing, then run a fresh final proof. diff --git a/PROVE-IT.md b/PROVE-IT.md index 4f85a2f..07b6905 100644 --- a/PROVE-IT.md +++ b/PROVE-IT.md @@ -48,13 +48,22 @@ Do not commit until you can answer: - The proof is recent - The diff matches the proof -After the checks and self-review are complete, stamp local proof for the git -gate: +After focused checks and self-review, run required broad verification through +the proof-stamping command for the git gate: ```bash node .codex/hooks/git-guard.cjs prove --reviewed ``` +For this repository, run and stamp the complete maintainer suite once with: + +```bash +node .codex/hooks/git-guard.cjs prove --reviewed --check "node scripts/run-proof-suite.cjs" +``` + +Do not run `node scripts/run-proof-suite.cjs` immediately before this command; +the proof-stamping invocation already runs it. + If setup has not detected proof commands yet, pass them explicitly: ```bash diff --git a/README.md b/README.md index c3c8a15..1b2646a 100644 --- a/README.md +++ b/README.md @@ -208,7 +208,7 @@ Layer 1: LOCAL TRUTH | Capability | Codex-specific shape | |------------|----------------------| | **Proof-aware git gates** | `git commit` and `git push` stay blocked until a fresh reviewed SDLC proof stamp is tied to the current repo content | -| **Codex-native review** | Uses `codex review --uncommitted`, `--base`, or `--commit`; mixed mode explicitly overrides review effort to keep its Sol review at `high` | +| **Codex-native review** | Reuses one stamped broad proof in a prompt-only Sol-high code review; predefined `--uncommitted`, `--base`, and `--commit` targets remain available for reviews without a custom prompt | | **Adaptive setup/update** | Default `npx` setup bootstraps first, then hands off into Codex for unresolved questions; update repairs drift without blind overwrites | | **Honest skill model** | `$sdlc` is the public repo-scoped workflow; helper skills stay support tooling instead of pretending Codex has slash commands | | **Cross-platform hook shape** | Universal Node hook entrypoints avoid Bash/PowerShell hook-config churn across macOS, Linux, Windows, and `type: module` repos | @@ -219,12 +219,18 @@ Layer 1: LOCAL TRUTH The git gate is proof-aware: `git commit` and `git push` are still hard manual checkpoints, but they can proceed when a fresh SDLC proof stamp exists. -After running the required checks and self-review, stamp proof: +After focused checks and self-review, run required broad verification through the proof-stamping command: ```bash node .codex/hooks/git-guard.cjs prove --reviewed ``` +For this repository's maintainer suite, use the single canonical command below. It runs all proof groups once and writes the receipt in that same invocation: + +```bash +node .codex/hooks/git-guard.cjs prove --reviewed --check "node scripts/run-proof-suite.cjs" +``` + If the repo has no detected commands in `.codex-sdlc/manifest.json`, provide the proof command explicitly: @@ -352,12 +358,17 @@ codex review --base main # Review one already-created commit codex review --commit + +# Proof-aware custom review: prompt only; do not add a predefined target flag +codex -c 'model_reasoning_effort="high"' review 'Review only this frozen diff. Base: . Candidate: . Proof: => . Do not rerun tests. Return prioritized code-review findings only.' ``` When `review_model = "gpt-5.6-sol"` is present, native Codex review uses Sol for the review pass. `review_model` does not set review reasoning independently: effort otherwise inherits the profile's global `model_reasoning_effort`. Mixed-mode agents must therefore use the explicit `high` override above (and the same prefix with `--base` or `--commit`) to provide the advertised Sol-high gate. Do not treat `/autoreview` as a required SDLC command. `auto_review` is a Codex approval-review setting for eligible tool approval prompts; it is not the code-diff review path. In yolo/full-bypass sessions, approval review usually does not apply because approvals are already bypassed. +Run one broad proof run total on the frozen candidate through the proof-stamping entrypoint. Do not run the suite directly and then rerun it through the guard. When supplying custom proof-aware instructions, use a prompt-only review. A custom prompt must not be combined with `--uncommitted`, `--base`, or `--commit`; those predefined target flags are for reviews without a custom prompt. Include the exact base identity, frozen candidate tree identity, proof command, and result, and say `Do not rerun tests`. Targeted verification is allowed only for a concrete suspected defect; never rerun the broad suite. + ## Repo-Scoped Skills `install.sh` and `setup.sh` scaffold repo-local Codex skills under `.agents/skills`. diff --git a/RELEASE.md b/RELEASE.md index 1c51288..7b89753 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -22,12 +22,14 @@ If the branch cannot be cleanly rebased or merged onto `origin/main`, stop and f ## 2. Required Proof Suite -Preferred path: run the bounded parallel proof runner and keep every check green before tagging: +Preferred path: run the bounded parallel proof runner once through the proof-stamping entrypoint and keep every check green before tagging: ```bash -node scripts/run-proof-suite.cjs +node .codex/hooks/git-guard.cjs prove --reviewed --check "node scripts/run-proof-suite.cjs" ``` +Do not run `node scripts/run-proof-suite.cjs` immediately before the command above; the proof-stamping invocation already runs it. + Use the serial fallback when debugging one failure at a time: ```bash diff --git a/SDLC-LOOP.md b/SDLC-LOOP.md index f8339c5..737abb8 100644 --- a/SDLC-LOOP.md +++ b/SDLC-LOOP.md @@ -23,7 +23,8 @@ Codex does not have a native `/sdlc` command. This file is the honest replacemen Commit coherent green slices after focused proof. Freeze the cumulative completion candidate and run one fresh broad proof before final review; relevant changes invalidate it and require a fresh final proof. 8. Review to a decision Review the full base-to-candidate diff once after it is stable. Severity ladder: P0 stops the line; P1 blocks completion; P2 is a bounded fix now or a follow-up issue; P3 never blocks and is recorded only when worthwhile. - Give reviewers the fresh proof command and result and say `Do not rerun tests`; missing or stale proof is a blocker to report, not permission to launch another broad suite. + Run one broad proof run total on the frozen candidate through `node .codex/hooks/git-guard.cjs prove --reviewed`; do not run the suite directly and then rerun it through the guard. + Use a prompt-only review when supplying custom proof-aware instructions. A custom prompt must not be combined with `--uncommitted`, `--base`, or `--commit`; those predefined target flags are for reviews without a custom prompt. Include the exact base identity, frozen candidate tree identity, proof command, and result and say `Do not rerun tests`. Targeted verification is allowed only for a concrete suspected defect; never rerun the broad suite. Missing or stale proof is a blocker to report, not permission to launch another broad suite. Reviewer role: inspect the frozen diff and return prioritized code-review findings only; do not edit, implement, run tests, re-plan, or perform follow-up work. The builder owns every correction through the normal SDLC loop. When two reviewers are required, they assess the same frozen candidate independently, exchange compact findings once, and return a joint ledger. Allow at most two corrective rounds. If P0/P1 remains, decompose, abandon, or escalate; never waive it or continue an unbounded review loop. Check every corrective finding against the base. If the blocker is candidate-born and outside the allowlist, remove that accretion instead of repairing it. diff --git a/lib/codex-config.sh b/lib/codex-config.sh index 104eb75..3b6c651 100644 --- a/lib/codex-config.sh +++ b/lib/codex-config.sh @@ -2,7 +2,7 @@ set -euo pipefail MINIMUM_GPT56_CODEX_VERSION="${MINIMUM_GPT56_CODEX_VERSION:-0.144.0}" -MODEL_POLICY_SCHEMA_VERSION=2 +MODEL_POLICY_SCHEMA_VERSION=3 require_gpt56_codex_version() { local version_output="" diff --git a/lib/refresh-manifest-hashes.cjs b/lib/refresh-manifest-hashes.cjs index e76710b..010fd6a 100644 --- a/lib/refresh-manifest-hashes.cjs +++ b/lib/refresh-manifest-hashes.cjs @@ -87,9 +87,6 @@ function synchronizeModelProfile(manifest, touchedFiles) { ...(manifest.model_profile || {}), selected_profile: selectedProfile, }; - if (Number.isInteger(profile.schema_version)) { - next.policy_schema_version = profile.schema_version; - } const baselineReasoning = profile.profiles?.[selectedProfile]?.main_reasoning; if (typeof baselineReasoning !== "string" || baselineReasoning === "") { throw new Error(`${profilePath} does not define main_reasoning for ${selectedProfile}`); diff --git a/setup.sh b/setup.sh index c8d081e..29267e4 100644 --- a/setup.sh +++ b/setup.sh @@ -1360,7 +1360,7 @@ const manifest = { }, model_profile: { selected_profile: process.env.MODEL_PROFILE_SELECTED || "", - policy_schema_version: Number(process.env.MODEL_POLICY_SCHEMA_VERSION_SELECTED || "2"), + policy_schema_version: Number(process.env.MODEL_POLICY_SCHEMA_VERSION_SELECTED || "3"), baseline_reasoning: process.env.REASONING_BASELINE_SELECTED || "high", escalation_reasoning: process.env.REASONING_ESCALATION_SELECTED || "xhigh", repo_risk_signals: process.env.REASONING_RISK_SIGNALS_SELECTED || "none detected during setup" diff --git a/skill-sources/sdlc/SKILL.template.md b/skill-sources/sdlc/SKILL.template.md index c4116f8..b4c5829 100644 --- a/skill-sources/sdlc/SKILL.template.md +++ b/skill-sources/sdlc/SKILL.template.md @@ -99,7 +99,7 @@ Use native Codex review for a second pass when the slice warrants it: `review_model` controls native Codex review model selection but does not set review reasoning independently. Mixed mode must use the explicit `high` command override above; apply the same prefix to `--base` or `--commit` reviews. This is a CLI review path, not a slash-command contract. -Pass the fresh proof command and result in the custom review prompt and say `Do not rerun tests`; the reviewer inspects the diff and evidence. Missing or stale proof is a blocker to report, not permission to launch another broad suite. +Run one broad proof run total on the frozen candidate through `node .codex/hooks/git-guard.cjs prove --reviewed`; do not run the suite directly and then rerun it through the guard. Use a prompt-only review when supplying custom proof-aware instructions. A custom prompt must not be combined with `--uncommitted`, `--base`, or `--commit`; those predefined target flags are for reviews without a custom prompt. Include the exact base identity, frozen candidate tree identity, proof command, and result and say `Do not rerun tests`. Targeted verification is allowed only for a concrete suspected defect; never rerun the broad suite. Missing or stale proof is a blocker to report, not permission to launch another broad suite. Reviewer role: inspect the frozen diff and return prioritized code-review findings only; do not edit, implement, run tests, re-plan, or perform follow-up work. The builder owns every correction through the normal SDLC loop. diff --git a/templates/AGENTS.baseline.md b/templates/AGENTS.baseline.md index af1ef71..9e17138 100644 --- a/templates/AGENTS.baseline.md +++ b/templates/AGENTS.baseline.md @@ -13,7 +13,8 @@ Read `TESTING.md` and `ARCHITECTURE.md` when present and relevant. If `GOALS.md` 5. Never claim completion without fresh proof. 6. Author-review and commit coherent green slices. Freeze the cumulative candidate for one fresh broad proof and completion review. 7. Severity ladder: P0 stops the line; P1 blocks completion; P2 is a bounded fix or follow-up issue; P3 never blocks. When two reviewers are required, they exchange compact findings once. Allow at most two corrective rounds; unresolved P0/P1 requires decomposition, abandonment, or escalation. - Give reviewers the fresh proof command and result and say `Do not rerun tests`; stale proof is a blocker to report, not permission to launch another broad suite. + Run one broad proof run total on the frozen candidate through `node .codex/hooks/git-guard.cjs prove --reviewed`; do not run the suite directly and then rerun it through the guard. + Use a prompt-only review when supplying custom proof-aware instructions. A custom prompt must not be combined with `--uncommitted`, `--base`, or `--commit`; those predefined target flags are for reviews without a custom prompt. Include the exact base identity, frozen candidate tree identity, proof command, and result and say `Do not rerun tests`. Targeted verification is allowed only for a concrete suspected defect; never rerun the broad suite. Stale proof is a blocker to report, not permission to launch another broad suite. Reviewer role: inspect the frozen diff and return prioritized code-review findings only; do not edit, implement, run tests, re-plan, or perform follow-up work. The builder owns every correction through the normal SDLC loop. If a blocker is candidate-born and outside the allowlist, remove that accretion instead of repairing it. diff --git a/templates/AGENTS.md.tmpl b/templates/AGENTS.md.tmpl index 120996f..c24d2bb 100644 --- a/templates/AGENTS.md.tmpl +++ b/templates/AGENTS.md.tmpl @@ -39,7 +39,8 @@ Use skills for the visible workflow contract, let hooks enforce silently, and ke 4. **Verify incrementally:** run focused proof for each coherent green slice; run the full required proof on the frozen completion candidate 5. **Active goals:** When `GOALS.md` exists, complete that active scope before claiming the run is done; do not confuse active goal completion with roadmap completion. 6. **Review to a decision:** author-review each incremental diff, then review the stable cumulative candidate once. Severity ladder: P0 stops the line; P1 blocks completion; P2 is a bounded fix or follow-up issue; P3 never blocks. When two reviewers are required, they exchange compact findings once. Allow at most two corrective rounds; unresolved P0/P1 requires decomposition, abandonment, or escalation. - - Give reviewers the fresh proof command and result and say `Do not rerun tests`; stale proof is a blocker to report, not permission to launch another broad suite. + - Run one broad proof run total on the frozen candidate through `node .codex/hooks/git-guard.cjs prove --reviewed`; do not run the suite directly and then rerun it through the guard. + - Use a prompt-only review when supplying custom proof-aware instructions. A custom prompt must not be combined with `--uncommitted`, `--base`, or `--commit`; those predefined target flags are for reviews without a custom prompt. Include the exact base identity, frozen candidate tree identity, proof command, and result and say `Do not rerun tests`. Targeted verification is allowed only for a concrete suspected defect; never rerun the broad suite. Stale proof is a blocker to report, not permission to launch another broad suite. - Reviewer role: inspect the frozen diff and return prioritized code-review findings only; do not edit, implement, run tests, re-plan, or perform follow-up work. The builder owns every correction through the normal SDLC loop. - If a blocker is candidate-born and outside the allowlist, remove that accretion instead of repairing it. diff --git a/tests/test-adapter.sh b/tests/test-adapter.sh index d8b76b0..28daeb8 100755 --- a/tests/test-adapter.sh +++ b/tests/test-adapter.sh @@ -4126,6 +4126,7 @@ const path = require("path"); const root = process.env.ROOT; const agents = fs.readFileSync(path.join(root, "AGENTS.md")); fs.writeFileSync(path.join(root, ".codex-sdlc", "manifest.json"), `${JSON.stringify({ + model_profile: { selected_profile: "mixed", policy_schema_version: 3 }, managed_files: { "AGENTS.md": `sha256:${crypto.createHash("sha256").update(agents).digest("hex")}`, ".codex-sdlc/model-profile.json": "sha256:old", @@ -4168,6 +4169,7 @@ NODE const manifest = require(process.env.MANIFEST_PATH); if (manifest.model_profile?.selected_profile !== "maximum") process.exit(1); if (manifest.model_profile?.baseline_reasoning !== "high") process.exit(1); +if (manifest.model_profile?.policy_schema_version !== 3) process.exit(1); NODE rm -rf "$legacy_dir" "$partial_dir" @@ -4989,6 +4991,8 @@ test_e2e_bypasses_hook_trust_only_for_ephemeral_automation() { test_docs_document_proof_stamp_gate() { if grep -q 'git-guard.cjs prove --reviewed' "$REPO_DIR/PROVE-IT.md" \ && grep -q 'git-guard.cjs prove --reviewed' "$REPO_DIR/README.md" \ + && grep -Fq 'node .codex/hooks/git-guard.cjs prove --reviewed --check "node scripts/run-proof-suite.cjs"' "$REPO_DIR/PROVE-IT.md" \ + && grep -Fq 'node .codex/hooks/git-guard.cjs prove --reviewed --check "node scripts/run-proof-suite.cjs"' "$REPO_DIR/README.md" \ && grep -q 'fresh SDLC proof' "$REPO_DIR/README.md" \ && grep -qi 'same-repository linked worktree' "$REPO_DIR/README.md" \ && grep -qi 'same-repository linked worktree' "$REPO_DIR/PROVE-IT.md" \ diff --git a/tests/test-packaging.sh b/tests/test-packaging.sh index 4fc8efa..c42e3d7 100644 --- a/tests/test-packaging.sh +++ b/tests/test-packaging.sh @@ -960,6 +960,10 @@ test_readme_documents_native_codex_review() { local explains_review_effort_boundary=true local explains_auto_review_boundary=true local avoids_autoreview_requirement=true + local has_single_proof_contract=true + local has_prompt_only_contract=true + local has_targeted_verification_boundary=true + local binds_base_and_candidate=true grep -q 'codex review' "$README" || has_review_command=false grep -q 'codex review --uncommitted' "$README" || has_uncommitted=false @@ -970,6 +974,11 @@ test_readme_documents_native_codex_review() { grep -Eqi 'review_model.*(does not|doesn.t).*reasoning|reasoning.*(does not|doesn.t).*review_model' "$README" || explains_review_effort_boundary=false grep -Eqi 'auto_review.*approval|approval.*auto_review' "$README" || explains_auto_review_boundary=false grep -Eqi '(must|always|requires).*/autoreview|/autoreview.*(must|always)' "$README" && avoids_autoreview_requirement=false + grep -Fqi 'one broad proof run total' "$README" || has_single_proof_contract=false + grep -Eqi 'prompt-only.*review|review.*prompt-only' "$README" || has_prompt_only_contract=false + grep -Eqi 'custom prompt.*(cannot|must not|do not).*--(uncommitted|base|commit)|(cannot|must not|do not).*--(uncommitted|base|commit).*custom prompt' "$README" || has_prompt_only_contract=false + grep -Eqi 'targeted verification.*concrete suspected defect|concrete suspected defect.*targeted verification' "$README" || has_targeted_verification_boundary=false + grep -Eqi 'Base: ]+>.*Candidate: ]+>' "$README" || binds_base_and_candidate=false if [ "$has_review_command" = "true" ] && [ "$has_uncommitted" = "true" ] && @@ -979,10 +988,14 @@ test_readme_documents_native_codex_review() { [ "$has_explicit_high_review" = "true" ] && [ "$explains_review_effort_boundary" = "true" ] && [ "$explains_auto_review_boundary" = "true" ] && - [ "$avoids_autoreview_requirement" = "true" ]; then - pass "README documents native Codex review without requiring unsupported autoreview slash commands" + [ "$avoids_autoreview_requirement" = "true" ] && + [ "$has_single_proof_contract" = "true" ] && + [ "$has_prompt_only_contract" = "true" ] && + [ "$has_targeted_verification_boundary" = "true" ] && + [ "$binds_base_and_candidate" = "true" ]; then + pass "README documents proof-aware native Codex review without redundant broad verification" else - fail "README does not document native Codex review and the auto_review boundary clearly enough" + fail "README does not document proof-aware native Codex review and its verification boundaries clearly enough" fi } diff --git a/tests/test-release.sh b/tests/test-release.sh index a8db263..b8281ad 100644 --- a/tests/test-release.sh +++ b/tests/test-release.sh @@ -7,6 +7,7 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" REPO_DIR="$SCRIPT_DIR/.." README="$REPO_DIR/README.md" RELEASE_DOC="$REPO_DIR/RELEASE.md" +PROVE_IT_DOC="$REPO_DIR/PROVE-IT.md" WORKFLOW="$REPO_DIR/.github/workflows/release.yml" UPSTREAM_SYNC_WORKFLOW="$REPO_DIR/.github/workflows/upstream-sync.yml" UPSTREAM_VERSION_FILE="$REPO_DIR/UPSTREAM_VERSION" @@ -270,19 +271,35 @@ test_release_checklist_enforces_sync_and_proof() { test_release_checklist_points_to_parallel_proof_runner() { local has_runner_command=true + local has_canonical_stamp_command=true local has_parallel_language=true local has_serial_escape_hatch=true grep -Fq 'node scripts/run-proof-suite.cjs' "$RELEASE_DOC" || has_runner_command=false + grep -Fq 'node .codex/hooks/git-guard.cjs prove --reviewed --check "node scripts/run-proof-suite.cjs"' "$RELEASE_DOC" || has_canonical_stamp_command=false grep -Eqi 'parallel|bounded jobs|jobs' "$RELEASE_DOC" || has_parallel_language=false grep -Fq 'node scripts/run-proof-suite.cjs --serial' "$RELEASE_DOC" || has_serial_escape_hatch=false if [ "$has_runner_command" = "true" ] && + [ "$has_canonical_stamp_command" = "true" ] && [ "$has_parallel_language" = "true" ] && [ "$has_serial_escape_hatch" = "true" ]; then - pass "RELEASE.md points maintainers to the parallel proof runner with a serial fallback" + pass "RELEASE.md stamps one parallel proof run and keeps a serial debugging fallback" else - fail "RELEASE.md does not document the parallel proof runner and serial fallback" + fail "RELEASE.md does not document the canonical stamped proof run and serial fallback" + fi +} + +test_proof_docs_do_not_pre_run_broad_checks() { + local valid=true + + grep -Fq 'After the checks and self-review are complete, stamp' "$PROVE_IT_DOC" && valid=false + grep -Fq 'After running the required checks and self-review, stamp proof:' "$README" && valid=false + + if [ "$valid" = "true" ]; then + pass "Proof docs route broad checks through the stamping command instead of pre-running them" + else + fail "Proof docs still tell maintainers to run broad checks before the stamping command" fi } @@ -371,6 +388,7 @@ test_readme_documents_maintainer_release_steps test_release_checklist_exists test_release_checklist_enforces_sync_and_proof test_release_checklist_points_to_parallel_proof_runner +test_proof_docs_do_not_pre_run_broad_checks test_parallel_proof_runner_exists_and_lists_full_suite test_readme_points_to_release_checklist test_roadmap_tests_are_version_agnostic diff --git a/tests/test-skill.sh b/tests/test-skill.sh index 211e932..7a8e5cd 100644 --- a/tests/test-skill.sh +++ b/tests/test-skill.sh @@ -482,6 +482,28 @@ test_sdlc_workflow_is_bounded_and_repairable() { fi } +test_sdlc_review_reuses_one_broad_proof() { + local file + local valid=true + + for file in "$REPO_SDLC_SKILL" "$SHIPPED_SDLC_SKILL" "$SDLC_LOOP" "$AGENTS_BASELINE" "$AGENTS_TEMPLATE"; do + grep -Fqi 'one broad proof run total' "$file" || valid=false + grep -Eqi 'prompt-only.*review|review.*prompt-only' "$file" || valid=false + grep -Eqi 'custom prompt.*(cannot|must not|do not).*--(uncommitted|base|commit)|(cannot|must not|do not).*--(uncommitted|base|commit).*custom prompt' "$file" || valid=false + grep -Eqi 'base.*candidate|candidate.*base' "$file" || valid=false + grep -Eqi 'targeted verification.*concrete suspected defect|concrete suspected defect.*targeted verification' "$file" || valid=false + grep -Eqi 'proof command and result|proof.*command.*result' "$file" || valid=false + done + + grep -Fq 'MODEL_POLICY_SCHEMA_VERSION=3' "$REPO_DIR/lib/codex-config.sh" || valid=false + + if [ "$valid" = "true" ]; then + pass "SDLC review consumes one proof receipt without rerunning broad suites" + else + fail "SDLC review does not consistently bind prompt-only review to one proof, base, candidate, and upgrade schema" + fi +} + test_skill_manifest_exists test_plugin_skill_resolves_bundled_scripts_from_plugin_root test_plugin_skill_handles_legacy_standalone_install @@ -499,6 +521,7 @@ test_repo_scoped_skills_are_codex_native test_repo_scoped_sdlc_skill_documents_codex_shape_and_repo_focus test_repo_scoped_sdlc_skill_documents_native_review test_sdlc_workflow_is_bounded_and_repairable +test_sdlc_review_reuses_one_broad_proof echo "" echo "=== Results: $PASSED passed, $FAILED failed ===" diff --git a/tests/test-update.sh b/tests/test-update.sh index daf875a..219c885 100644 --- a/tests/test-update.sh +++ b/tests/test-update.sh @@ -1581,9 +1581,12 @@ NODE check_output=$(run_check "$ws") || valid=false grep -Fq 'Default to `high`' "$ws/SDLC-LOOP.md" || valid=false + grep -Fqi 'one broad proof run total' "$ws/AGENTS.md" || valid=false grep -Fq 'USER CUSTOM POLICY' "$ws/START-SDLC.md" || valid=false grep -Fq 'Use xhigh reasoning by default for this repo.' "$ws/START-SDLC.md" || valid=false grep -Fq 'model_reasoning_effort="high"' "$ws/.agents/skills/sdlc/SKILL.md" || valid=false + grep -Fqi 'one broad proof run total' "$ws/.agents/skills/sdlc/SKILL.md" || valid=false + grep -Eqi 'prompt-only.*review|review.*prompt-only' "$ws/.agents/skills/sdlc/SKILL.md" || valid=false grep -Fq 'USER CUSTOM SETUP HELPER' "$ws/.codex-home/skills/setup-wizard/SKILL.md" || valid=false grep -Fq 'USER CUSTOM UPDATE HELPER' "$ws/.codex-home/skills/update-wizard/SKILL.md" || valid=false json_text_equals "$(cat "$ws/.codex-sdlc/manifest.json")" 'data.managed_files[".agents/skills/sdlc/SKILL.md"].startsWith("sha256:")' "true" || valid=false @@ -1655,7 +1658,7 @@ EOF [ "$(cat "$ws/.codex-sdlc/model-profile.json")" = "$profile_before" ] || valid=false echo "$output" | grep -Fq '.codex-sdlc/model-profile.json: customized -> skip' || valid=false json_text_equals "$check_output" 'data.managed_files[".codex-sdlc/model-profile.json"].status' "customized" || valid=false - json_text_equals "$(cat "$ws/.codex-sdlc/manifest.json")" 'data.model_profile.policy_schema_version' "2" || valid=false + json_text_equals "$(cat "$ws/.codex-sdlc/manifest.json")" 'data.model_profile.policy_schema_version' "3" || valid=false echo "$second_output" | grep -Fq 'No changes applied.' || valid=false echo "$second_output" | grep -Fq 'refresh model policy' && valid=false echo "$second_output" | grep -Fq 'refresh generated model policy' && valid=false @@ -1714,7 +1717,7 @@ NODE output=$(run_update "$ws") || valid=false second_output=$(run_update "$ws") || valid=false - json_text_equals "$(cat "$ws/.codex-sdlc/manifest.json")" 'data.model_profile.policy_schema_version' "2" || valid=false + json_text_equals "$(cat "$ws/.codex-sdlc/manifest.json")" 'data.model_profile.policy_schema_version' "3" || valid=false [ "$(cat "$ws/.codex-sdlc/model-profile.json")" = "$profile_before" ] || valid=false [ "$(cat "$ws/AGENTS.md")" = "$agents_before" ] || valid=false [ "$(cat "$ws/SDLC-LOOP.md")" = "$loop_before" ] || valid=false diff --git a/update.sh b/update.sh index dbd4e68..ac5b3de 100644 --- a/update.sh +++ b/update.sh @@ -416,6 +416,10 @@ case "$MANIFEST_MODEL_POLICY_SCHEMA_VERSION" in ;; esac MODEL_PROFILE_MIGRATION="$MODEL_PROFILE_DEFAULTED" +if [ "$MODEL_POLICY_SCHEMA_MIGRATION" = "true" ]; then + MODEL_PROFILE_MIGRATION=true + RECORD_MODEL_POLICY_MIGRATION=true +fi if [ "$MODEL_PROFILE_METADATA_STATUS" = "missing" ]; then MODEL_PROFILE_MIGRATION=true elif [ "$MODEL_PROFILE_METADATA_STATUS" = "match" ] && model_profile_metadata_needs_refresh ".codex-sdlc/model-profile.json" "$MODEL_PROFILE"; then