chore: retire tools/change_gate and the agentic-review route superseded by hw-gate - #700
chore: retire tools/change_gate and the agentic-review route superseded by hw-gate#700Kaden-Schutt wants to merge 1 commit into
Conversation
hw-gate sol prelimsummary: Retires the superseded change_gate and agentic-review implementations, their tests, configuration, and agent skills; updates contributor and validation documentation to point exclusively at hw-gate; and removes the deleted change_gate unittest modules from no-GPU CI. No inference-engine, model-loading, serving, kernel, dependency, build, or workflow behavior is changed. run_hardware: true routes:
unavailable_routes: (none) |
hw-gate evidence — 2 lane(s) — verdict passlane hiptrx (gfx1201)hw-gate evidence
fixturesno fixtures kernelnot run lane hipx (gfx1100)hw-gate evidence
fixturesno fixtures kernelnot run |
hw-gate sol verdict{
"claim_verdict": "not-exercised",
"confidence": 0.94,
"coverage": {
"gaps": [
"The hw-gate evidence contains successful hiptrx and hipx builds but no execution of scripts/no-gpu-ci.sh, so the changed CI script path was not behaviorally exercised.",
"scripts/leanup-thresholds.txt is a policy-file change and requires human review under the hard floor."
],
"surfaces_evidenced": [
"docs",
"build"
],
"surfaces_touched": [
"docs",
"ci",
"scripts",
"policy"
]
},
"decision": "needs-human",
"eyeball": [
"A human should review scripts/leanup-thresholds.txt and confirm that replacing \u201cchange_gate routes\u201d with \u201cCI gates\u201d preserves the intended ratchet policy.",
"A human should inspect scripts/no-gpu-ci.sh and, ideally, run it once to confirm removal of the retired unittest modules leaves the remaining no-GPU CI battery intact."
],
"phase": "verdict",
"rationale": "Both hardware lanes built head 28739bd6ef00da4130fced8d9b32e314ded60df1 successfully with identical daemon and hipfire MD5s, and no load, serve, kernel, model-artifact, or GPU-byte surface changed. No regression is evidenced. However, hw-gate ran no buckets or fixtures and did not exercise scripts/no-gpu-ci.sh; therefore the author's CI-integrity claim is not exercised by the supplied hardware evidence. The change to scripts/leanup-thresholds.txt is independently a policy hard-floor item, requiring needs-human.",
"regressions": []
}Floor: hard=['policy_paths: scripts/leanup-thresholds.txt'] soft=["coverage_gaps: ['The hw-gate evidence contains successful hiptrx and hipx builds but no execution of scripts/no-gpu-ci.sh, so the changed CI script path was not behaviorally exercised.', 'scripts/leanup-thresholds.txt is a policy-file change and requires human review under the hard floor.']", 'model needs-human'] model_decision=needs-human final=needs-human |
There was a problem hiding this comment.
hw-gate sol verdict needs-human: Both hardware lanes built head 28739bd successfully with identical daemon and hipfire MD5s, and no load, serve, kernel, model-artifact, or GPU-byte surface changed. No regression is evidenced. However, hw-gate ran no buckets or fixtures and did not exercise scripts/no-gpu-ci.sh; therefore the author's CI-integrity claim is not exercised by the supplied hardware evidence. The change to scripts/leanup-thresholds.txt is independently a policy hard-floor item, requiring needs-human.
|
announcement: Hold, for the policy floor only: scripts/leanup-thresholds.txt is a policy path and needs a maintainer's human-reviewed label, though the hunk is a single comment word and the ratchet still asserts 21 metrics with 0 violations. I closed the gap Sol flagged by running every Python line of scripts/no-gpu-ci.sh verbatim on your head and on master: the retained redline unittest modules run, the deleted packages are genuinely gone, and autoresearch/ar/tests collects cleanly without review_fixtures. The battery is red on both trees with byte-identical failures (test_mq4c_repack.py, test_golden, check-env-docs.py) — pre-existing on master, none in files you touched, and the passed-count drop is exactly the 457 review tests you deleted. Nothing live references the retired route anymore, and the shipped binaries match base apart from the embedded commit hash. investigation: | question | route | result | evidence | |---|---|---|---| | Does anything live in the tree still import or exec the deleted packages (tools.change_gate, autoresearch.ar.review, review_fixtures, the .agents skills, .github/agentic-review config)? | grep -rn 'change_gate|change-gate|agentic[-_]review|agentic-pr|ar.review|ar/review|providers.local|capabilities-v1|trusted-publishers' (whole tree, gitignored included), then a targeted pass over .github/ .agents/ scripts/ (incl. scripts/hw-gate/) tools/ autoresearch/ tests/ CLAUDE.md AGENTS.md | Zero live references. The only hits are historical: docs/governance/2026-08-16-phase3-scope.md:75,87,236; docs/superpowers/specs/2026-07-15-agentic-pr-review-workflow-design.md:170,209,290; experiments/flash-attn-ck-sidecar/results/asym3_givens_pp8192_master_20260827/README.md:69; and a doc-comment at crates/rdna-compute/src/attention.rs:3037 citing a design-doc filename. scripts/hw-gate/{select,run,review,merge_evidence}.py import nothing from the deleted packages. | fable-summary.md § Residual references | | Sol's gap: does the changed scripts/no-gpu-ci.sh path actually run on the PR head — deleted modules gone, retained modules intact, pytest sweep still collects autoresearch/ar/tests without review_fixtures? | bash -n scripts/no-gpu-ci.sh; python3 -c 'import yaml' parse of all 4 workflows; python3 -c 'import tools.change_gate' / 'import autoresearch.ar.review' / 'import autoresearch.ar.tests.review_fixtures'; python3 -m unittest tools.redline.tests.test_product_bench tools.redline.tests.test_golden tools.redline.tests.test_serve_diff tools.redline.tests.test_lower tools.redline.tests.test_dispatch_profile; python3 scripts/test_install_revision.py; python3 scripts/test_uninstall.py; python3 scripts/check-env-docs.py; python3 -m pytest tests scripts/test_astrea.py autoresearch/ar/tests (each verbatim from no-gpu-ci.sh:17,25-30) | bash -n ok; no-gpu-ci.yml/ci.yml/hw-gate.yml/registry.yml parse. All three deleted imports raise ModuleNotFoundError (deletion is complete, no stale pycache shadow). All 5 retained redline modules import; unittest Ran 179 → FAILED (failures=2, errors=3), all in test_golden; test_install_revision OK (14), test_uninstall OK (6); check-env-docs.py EXIT=1; pytest 5 failed / 378 passed / 4 skipped, all 5 failures in tests/test_mq4c_repack.py. autoresearch/ar/tests collects cleanly with review_fixtures.py gone. | no-gpu-ci-pytest-head.log | | Are the reds in that battery caused by this PR or pre-existing on master? | Same pytest sweep, same unittest line, and same check-env-docs.py run in the base checkout at e23c55e; diff of FAILED/ERROR names and of check-env-docs output head vs base; git diff --stat base..HEAD -- scripts/mq4c_repack.py tests/test_mq4c_repack.py tools/redline scripts/check-env-docs.py crates; pytest --collect-only on the deleted test files at base | IDENTICAL on all three: the same 5 test_mq4c_repack.py FAILED names, the same 5 test_golden FAIL/ERROR names (Ran 179, failures=2, errors=3), byte-identical check-env-docs.py output with EXIT=1. The diff touches none of those files (diff --stat empty). Base passes 835 vs head 378; the deleted test files collect exactly 457 tests on base, and 835 − 457 = 378, so every lost test is one the PR intentionally deleted. No regression. | deleted-test-count-base.txt | | Does the policy-file edit (scripts/leanup-thresholds.txt) change anything the ratchet reads? | git diff base..HEAD -- scripts/leanup-thresholds.txt; bash scripts/leanup-ratchets.sh . on head | The hunk is one comment line (leanup-thresholds.txt:33, 'change_gate routes' → 'CI gates'); no metric line changes. Ratchet on head: OK — 21 metric(s) asserted, 0 violations; ungated_examples 48 against ceiling 48, unchanged. | leanup-ratchets-head.log | | Is 'no code under crates/ or kernels/ changes' true at the binary level? | md5sum of daemon, hipfire, hipfire-detect, hipfire-quantize in $HW_GATE_BIN vs $HW_GATE_BASE_BIN; strings + --version on both hipfire binaries | daemon (17ba7dfa…), hipfire-detect (05e11024…), hipfire-quantize (cf19a3ff…) byte-match base. hipfire differs (d21e2a73 vs 5919e558) solely because crates/hipfire-cli/build.rs:34-40 embeds the commit and ref: 'hipfire 0.3.0 (28739bd; detached)' vs '(e23c55e; master)'. No Rust behavior changed. | binary-md5-head-vs-base.txt | unproven:
rationale: The hard floor fired on policy_paths (scripts/leanup-thresholds.txt), so the decision space is hold or block; hold is correct because I found no evidence failure. The policy hunk is one comment word at scripts/leanup-thresholds.txt:33 with no metric change, and scripts/leanup-ratchets.sh on head reports 21 asserted / 0 violations with ungated_examples 48 == ceiling. Sol's coverage gap (no-gpu-ci.sh never exercised) is now closed: I ran every Python line of scripts/no-gpu-ci.sh:17,25-30 verbatim on head and again on base. The changed unittest line (scripts/no-gpu-ci.sh:25) runs its five retained tools.redline modules; the three deleted change_gate modules and autoresearch.ar.review raise ModuleNotFoundError; autoresearch/ar/tests collects without review_fixtures.py. The battery is red on head — but every FAILED/ERROR name and the check-env-docs output are byte-identical on master e23c55e, and the diff touches none of the implicated files, so nothing regressed. The 835→378 passed delta is exactly the 457 tests the PR deletes. No live reference to the retired route remains anywhere in .github/, .agents/, scripts/ (including scripts/hw-gate/), tools/, autoresearch/, or tests/; the eight residual mentions are historical docs/specs/an experiment README/a code comment citing a design-doc filename. Binary A/B: daemon, hipfire-detect, hipfire-quantize byte-match base; hipfire differs only by the embedded commit+ref from crates/hipfire-cli/build.rs:34-40. What changes this to a merge: a maintainer applying human-reviewed after reading leanup-thresholds.txt:33 and the reworded .github/PULL_REQUEST_TEMPLATE.md:60 / CONTRIBUTING.md:106-107 lines. Evidence: $HW_GATE_EVIDENCE/fable-summary.md and the seven files it indexes. hard floor: ['policy_paths: scripts/leanup-thresholds.txt'] soft floor: ["coverage_gaps: ['The hw-gate evidence contains successful hiptrx and hipx builds but no execution of scripts/no-gpu-ci.sh, so the changed CI script path was not behaviorally exercised.', 'scripts/leanup-thresholds.txt is a policy-file change and requires human review under the hard floor.']", 'model needs-human'] |
There was a problem hiding this comment.
Hold, for the policy floor only: scripts/leanup-thresholds.txt is a policy path and needs a maintainer's human-reviewed label, though the hunk is a single comment word and the ratchet still asserts 21 metrics with 0 violations. I closed the gap Sol flagged by running every Python line of scripts/no-gpu-ci.sh verbatim on your head and on master: the retained redline unittest modules run, the deleted packages are genuinely gone, and autoresearch/ar/tests collects cleanly without review_fixtures. The battery is red on both trees with byte-identical failures (test_mq4c_repack.py, test_golden, check-env-docs.py) — pre-existing on master, none in files you touched, and the passed-count drop is exactly the 457 review tests you deleted. Nothing live references the retired route anymore, and the shipped binaries match base apart from the embedded commit hash. The hard floor fired on policy_paths (scripts/leanup-thresholds.txt), so the decision space is hold or block; hold is correct because I found no evidence failure. The policy hunk is one comment word at scripts/leanup-thresholds.txt:33 with no metric change, and scripts/leanup-ratchets.sh on head reports 21 asserted / 0 violations with ungated_examples 48 == ceiling. Sol's coverage gap (no-gpu-ci.sh never exercised) is now closed: I ran every Python line of scripts/no-gpu-ci.sh:17,25-30 verbatim on head and again on base. The changed unittest line (scripts/no-gpu-ci.sh:25) runs its five retained tools.redline modules; the three deleted change_gate modules and autoresearch.ar.review raise ModuleNotFoundError; autoresearch/ar/tests collects without review_fixtures.py. The battery is red on head — but every FAILED/ERROR name and the check-env-docs output are byte-identical on master e23c55e, and the diff touches none of the implicated files, so nothing regressed. The 835→378 passed delta is exactly the 457 tests the PR deletes. No live reference to the retired route remains anywhere in .github/, .agents/, scripts/ (including scripts/hw-gate/), tools/, autoresearch/, or tests/; the eight residual mentions are historical docs/specs/an experiment README/a code comment citing a design-doc filename. Binary A/B: daemon, hipfire-detect, hipfire-quantize byte-match base; hipfire differs only by the embedded commit+ref from crates/hipfire-cli/build.rs:34-40. What changes this to a merge: a maintainer applying human-reviewed after reading leanup-thresholds.txt:33 and the reworded .github/PULL_REQUEST_TEMPLATE.md:60 / CONTRIBUTING.md:106-107 lines. Evidence: $HW_GATE_EVIDENCE/fable-summary.md and the seven files it indexes.
…ed by hw-gate Follow-up named in #679: hw-gate is the review rung, so the pre-hw-gate route goes. Deleted tools/change_gate (13 files), autoresearch/ar/review + its tests (22), .github/agentic-review (4), .agents/skills/agentic-pr-{discovery,static-review,review} (10), and tests/test_agentic_pr_review_skills.py. References retargeted at the hw-gate request block / docs/VALIDATION.md § hw-gate: PULL_REQUEST_TEMPLATE.md, CONTRIBUTING.md, docs/VALIDATION.md, scripts/leanup-thresholds.txt; scripts/no-gpu-ci.sh drops the three change_gate unittest modules (the redline modules it still runs all exist). .gitignore loses the dead providers.local.json entry. Left in place on purpose: docs/governance and docs/perf-checkpoints (immutable ledgers), dated plan/spec/result records under docs/, and the unrelated `autoresearch` kernel-loop package.
28739bd to
36dd9aa
Compare
…t's raw output on a no-decision Run 33866758629 (warpfront#702) ended the decide phase in 10 s with "omp decide: no JSON object in assistant text", and the uploaded fable-evidence/ contained warpfront#700's fable-summary.md and warpfront#686's route outputs: the workflow does `mkdir -p fable-evidence fable-home` in a reused runner workspace, so every session inherits the previous PR's files and can cite them as its own. `rm -rf` both before the mkdir. On the no-JSON path review.py discarded the assistant text it had already extracted, so the artifact carried nothing to diagnose the failure with. decision.json now records `fable_error` and `fable_raw` {assistant_text_tail, stderr_tail}; the step log gets the tail too. 103/103 in scripts/hw-gate/tests; workflow YAML parses.
Summary
The follow-up named in #679's after-merge list: hw-gate is the review rung, so the pre-hw-gate route goes.
Deleted (50 files):
tools/change_gate(13),autoresearch/ar/review+ tests (22),.github/agentic-review(4),.agents/skills/agentic-pr-{discovery,static-review,review}(10),tests/test_agentic_pr_review_skills.py.Retargeted at the hw-gate request block /
docs/VALIDATION.md§ hw-gate:PULL_REQUEST_TEMPLATE.md,CONTRIBUTING.md,docs/VALIDATION.md,scripts/leanup-thresholds.txt.scripts/no-gpu-ci.shdrops the threechange_gateunittest modules (the redline modules it still runs all exist;bash -nclean)..gitignoreloses the deadproviders.local.jsonentry.Left in place on purpose:
docs/governance/**anddocs/perf-checkpoints/**(immutable ledgers), dated plan/spec/result records underdocs/, and the unrelatedautoresearchkernel-loop package. Residual mentions: 7 lines, all historical.Which surface(s) does this touch?
.github/, no-gpu-ci) — human mergeEvidence
rg -n 'change_gate|change-gate|agentic-review|agentic_review|agentic-pr|autoresearch' --hidden -g '!.git'→ only the 7 historical lines above..github/workflows/no-gpu-ci.ymlunchanged and parses; every unittest modulescripts/no-gpu-ci.shstill names exists.No code under
crates/orkernels/changes; no hardware surface.