Repository navigation
[Experimental] Second semantic-normalization smoke pin - AB#652973 (no product merge/release) - #940
Draft
Wenjie Fan (gggdttt) wants to merge 1 commit into
Conversation
AB#652973. Preparation only above #938 at 646e280. Replace only the action default engine SHA and matching workflow test constant with 7c7c12c2188dd7bee063f925b9d055d05075beff. No product merge or release intent; no evaluation dispatch authorized. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Wenjie Fan (gggdttt)
added this pull request to stack #939
October 6, 2026 08:57
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PREP ONLY: experimental second semantic-normalization smoke pin
AB#652973. No product merge or release intent. NOT_AUTHORIZED for execution. This prepares the third layer above #938; it is not a standalone change against
main, a retry of the first combined smoke, or evidence of HTTP A/B causality. No workflow dispatch, reviewer generation, judge invocation, merge, or result publication was performed as part of this preparation.Base:
gggdttt-support-normalization-manifestsat exact #938 head646e2802bc89993e048c410a6f146bad3fc8237e. Initial HEAD, remote branch, and open draft #938 matched that SHA, with a clean worktree. #938 still targets #911 branchgggdttt-http-knowledge-ab-gateat0cbb709caaf92fcb2017bcfa0b1c77185d37cbc6. Both lower PRs are untouched. Existing native stack 939 was read-only and remained ordered[911, 938]; the coordinator owns any guarded append of this PR. No stack metadata was created, recreated, or mutated here.Head:
5cacfe09231352d152d8827c307585e85b15c391ongggdttt-prepare-normalized-smoke-stack.Exactly two substitutions
Replace
da9236a6f43d47c37e0bd59ebced33b6fe73f277with7c7c12c2188dd7bee063f925b9d055d05075beffexactly once in each file:.github/actions/install-agent-harnesses/action.yml: the default engine SHA inENGINE_SHA.tests/test_review_workflows.py:DEFAULT_ENGINE_SHA.No changes to uploader support, manifest models, dataset/gold, fixtures, runtime, scorer, adapter, package version (
0.14.0.dev1), CLI versions, model choices, workflow inputs, or publication gates. #938's strict normalization consumer and the two exact diagnostic selectors**/_review-report.raw.jsonand**/_review-source-bounds.jsonare inherited unchanged. All 14 existing diagnostic selectors, their order, seven-day diagnostic retention, and raw-OTel/broad-checkout exclusions are preserved.Immutable transitive provenance
646e2802bc89993e048c410a6f146bad3fc8237e5cacfe09231352d152d8827c307585e85b15c3917c7c12c2188dd7bee063f925b9d055d05075beff4a0885e207cf96617b33d54d0146debbcf7d98418c9385c16fdb1f21afa6ce957420ead73294d06d1.6Recursive engine tree comparison returned all 44 non-directory entries, without truncation: 43 identical mode/type/blob entries; only
agents/ALReviewAgent/bcquality.config.yamlchanged. Its bytes are exactly the parent config with the single refb74967bc5b7a454eae19d6a1250199afd869f064replaced by8c9385c16fdb1f21afa6ce957420ead73294d06d; version1.6and every other byte are unchanged. Engine tree:29a070d5a3106b88f7fc919f3d6569f9ad593bad. Config blob:54239fecf163a2c90003a0a559f6350826cc8aed->e2e68b42eacbcaa8693e26578a9d9e52bf7a7052.Recursive BC-Bench tree proof: 937 entries, identical path set and modes, 935 unchanged mode/type/blob entries, and exactly the two one-SHA substitutions above. No other tracked entry differs from #938.
9ba9c56ab8f9d0a28fb1ab563a4a682ac722b361382170b3f0ba11c3e05c573abbae272217bdcd06c31d48fe99fdd6b7fa1b4be42f46ccc71d5fcec3->83a3b9afe61a59d75e6724c9c6298556679eaf11bbaabb76432c78ae3d642353571da2b432adddcd->568a1aadbc074e10bdca5ecd46c2fad078b577cc027fcf04cdbf506aeada64ef70498b059006d946e6804162a662ccbba45ef1d82e5d94c90fbbbbf98a37b2c0eb636dd6a96b1a1934bdbd7f4068e9a75b05639a(1,413 bytes).Canonical diff command:
git diff --no-ext-diff --no-textconv --no-color --no-renames --full-index --binary --src-prefix=a/ --dst-prefix=b/ 646e2802bc89993e048c410a6f146bad3fc8237e 5cacfe09231352d152d8827c307585e85b15c391 --. Hash raw stdout bytes, without PowerShell re-encoding.Local validation
uv run --no-sync pytest -q tests/test_codereview_fixtures.py tests/test_review_workflows.py tests/test_pr_review_run_manifest.py tests/test_pr_review_agent.py tests/test_pr_review_metrics.py tests/test_pr_review_metrics_reporting.py tests/test_pr_review_output.py tests/test_codereview.py: 446 passed. This includes [Experimental support] Accept strict normalization manifests #938's legacy, ID-only, range-only, combined 4+4, malformed/lifecycle/strictness, omission/roundtrip, mocked adapter/evaluator, and exact diagnostic inclusion/exclusion tests, plus all HTTP fixture/workflow checks (the expanded equivalent of the earlier 95+ focused suite).uv run --no-sync pytest -q: 1234 passed, 2 skipped, 1 deselected.ty check . --ignore=unresolved-import --exclude notebooks/passed with the same pre-existing unused-ignore warning in untouchedsrc/bcbench/redteam.py:150.uv run --no-sync pre-commit run --all-files: all 11 hooks passed. Initial Windows child-process propagation of emptycore.fsmonitorfailed; rerun replaced only that command-local environment value with equivalentfalse. No repository/global config change, skipped hook, or unrelated code edit.git diff --checkand staged diff checks passed. Commit has the required Copilot co-author trailer.These are deterministic local checks, not a new smoke result, live model validation, or a claim of complete remote CI.
Fresh proposed one-arm authorization — NOT GRANTED
A new explicit authorization would apply only to this exact Bench head, engine, and BCQuality chain, and only to the following ordered seven-entry matrix:
One arm, one run: at most 7 entry generations, 140 reviewer processes (19 ordered leaves + 1 root per entry), and 5 gold-judge invocations. There are 8 expected comments across 5 judge-eligible entries. These are invocation bounds, not provider-call/token/credit/dollar caps. No retries, replacements, replications, extra adjudicator, publication, release, or product merge. A failed/missing/invalid result remains a failed observation; stop rather than spending replacement budget or starting a second run.
Freeze CLI
1.0.88, rootgpt-5.6-sol, leafgpt-5.6-luna, gold judgegpt-5.3-codex, serial leaves, configured concurrency4/ effective1, 30-minute CLI timeout, and both severity floorsMedium. Any separately authorized dispatch must resolve to Bench head5cacfe09231352d152d8827c307585e85b15c391, use the exactentriesarray, explicitengine-sha=7c7c12c2188dd7bee063f925b9d055d05075beff,test-run=true,modified-only=false,repeat=1, and preserve the existing concurrency group/matrix cap. Revalidate all heads, clean fetched checkouts, runtime identities, dataset/fixture hashes, and source provenance before spending. This retains the no-tag/no-requeue route and summarymock=true: no Braintrust, Kusto, or leaderboard writes. Summary mock does not mock reviewer generation or the gold judge. Diagnostic artifacts remain allowed for the required audit.Primary objective: 7/7 valid terminal reports with zero semantic-contract failures, not merely manifest completion or schema validity. Audit every original raw leaf report, its recorded SHA-256, accepted leaf report, normalization manifest, and
_review-source-bounds.json; verify every ID/range transformation is allowed, exactly recorded and source-bounded, and that findings/evidence are neither silently dropped nor rewritten. Check root/nested outcomes, all ordered process roles, observed models, complete usage, and executed/declared pin equality. The inherited consumer validates audit metadata shape, not raw-byte integrity or source-bound contents: those require the explicit audit.Preserve all three positive controls: consumed-TryFunction (
synthetic__errh-tryfunction-swallowed-01), swallowed HTTP execution failure (synthetic__http-consumed-false-01), and completed non-2xx body processing (synthetic__http-status-body-01). Inventory unrelated findings and report raw versus accepted findings transparently. This second combined semantic-normalization smoke is not an HTTP knowledge A/B comparison or a causal claim; do not pool it with or reuse authorization from earlier attempts.PREPARATION ONLY / NO PAID EXECUTION AUTHORIZED.