[Experimental] Correct HTTP clean fixture and prepare v4 smoke head - AB#652984

[Wenjie Fan (gggdttt)]

PREP ONLY: V4 clean fixture + bounded generation-schema follow-up

AB#652984 (child task); context Bug AB#652973. Experimental DRAFT. NOT_AUTHORIZED for another execution. No product merge or release intent. A normal follow-up commit retains the V4 fixture correction, replaces the engine pin and adds exactly two diagnostic selectors. No amend/force-push, new smoke/model/judge invocation, or native stack metadata mutation.

Current CI gate is BLOCKED. CI run 37594877366 failed both lint-and-test jobs in the unrelated ty-bcbench-core hook. GitHub's stack CI tested synthetic merge dc8e34aeedbb9329cfc41045ee2438e22602de8c of this head into ultimate main base e506dd15df0ba98c62869b8bd944d7e1d8443877, not just this pinned branch. ty 0.0.85 reports truthiness-test-of-none-union at packages/bcbench-core/src/bcbench_core/bc.py:92 and logs.py:60. Both file blobs and .pre-commit-config.yaml are identical to that main base; these core files/hook are absent from this branch. This is not a failure in the four changed paths. It is documented, not fixed, bypassed or rerun here; no claim of all-green CI or readiness to execute.

Identity	Exact value
Current PR941 head	e15e5adbc31754e222b9d2e97057fe9c8c2fb9f6
Current tree	70bfaf313f2f46a795567143f0a36b784a6a11dc
Head branch	gggdttt-clean-optional-http-fixture
Unchanged base	gggdttt-prepare-normalized-smoke-stack / #940
Unchanged base SHA	5cacfe09231352d152d8827c307585e85b15c391
Previous PR941 head / direct parent	184d58f19feaadf2b20e7d9f35a10b164f8cf6d4
Native stack 939	[911,938,940,941], metadata unchanged

Live PR941 was open/draft at its expected old head before editing and normal push. Lower heads remain #911 0cbb709caaf92fcb2017bcfa0b1c77185d37cbc6, #938 646e2802bc89993e048c410a6f146bad3fc8237e, and #940 5cacfe09231352d152d8827c307585e85b15c391.

V4 failure and new engine

The separately authorized V4 run 37586436198 stopped with 5/7 valid terminal observations, one semantic-consumer failure, and one cancellation-incomplete entry. Its authorization is consumed; its result/receipt/approval cannot authorize or substitute for another run.

synthetic__privacy-015's Error Handling leaf emitted cumulative rather than per-file coordinates: line 58 in a 29-line CustomerDataExporter file, and line 98/range 87-99 in a 42-line ExternalCRMSync file. Multi-file nl -ba continues numbering across operands. The engine correctly rejected these out-of-bounds locations rather than normalizing them, emitted an explicit failed-leaf stub/partial root, and BC-Bench correctly rejected the partial manifest before judging. One cancellation stopped status-body after the GitHub step failure became observable; no retry/replacement occurred. Optional-clean was correctly silent; consumed-false matched; status-body was not judged. This is failed reliability evidence, not successful smoke or HTTP A/B causality.

The new upstream engine bounds leaf and nested-leaf generation schemas by exact source files and per-file line/start/end maxima. Its prompt explicitly restarts numbering per file, forbids multi-operand nl -ba, and supplies complete inline limits or the full bounds artifact. These schemas are model-readable contracts, not constrained decoding or a generation guarantee. No coordinate conversion, line clamping, location deletion or model retry is introduced; invalid reports fail closed.

Exact follow-up changes

• Replace 7c7c12c2188dd7bee063f925b9d055d05075beff with 0b01ed8c90f3b07cbc693aa8869f84a16ded338d in exactly the two executable pins: .github/actions/install-agent-harnesses/action.yml (ENGINE_SHA) and tests/test_review_workflows.py (DEFAULT_ENGINE_SHA).
• Add exactly ${{ env.EVALUATION_RESULTS_DIR }}/**/_review-findings-report.leaf.schema.json and ${{ env.EVALUATION_RESULTS_DIR }}/**/_review-findings-report.root.schema.json after the existing source-bounds selector.
• Freeze the complete ordered 16-selector list, uniqueness, seven-day retention and warning behavior; add nine positive/negative selection cases for exact root/nested schema files and exclusion of backups, unknown roles, generic/arbitrary schemas. Existing raw-OTel/checkout exclusions remain.
• Update existing docs/code-review.md with failed-run context, new provenance, generated-schema audit and proposed fresh authorization.

All inherited 14 selectors remain once in their original relative order. Diagnostic/result retention, always(), no-publish gates, concurrency, CLI/model/severity/judge inputs and the entire workflow outside the two inserted lines remain unchanged. The new schema artifacts enable independent checking of actual generation contracts, not only the shared BCQuality schema.

No dataset/fixture, manifest-model, runtime, scorer, adapter, package-version, CLI-version, model-choice, severity or judge change in this follow-up.

Replacement-engine provenance

Component	Revision
Replacement engine	0b01ed8c90f3b07cbc693aa8869f84a16ded338d
Sole direct parent / microsoft/BC-ALAgents#80 head	84189e8ae262320a7a71b1cf75a63aaedbb40f2b
Engine tree	736e8c6b58e6def6077d7b538b09ae43b093788b
Engine-owned microsoft/BCQuality#220 head	8c9385c16fdb1f21afa6ce957420ead73294d06d
BCQuality version	1.6

Complete recursive remote comparison: 45 non-directory entries, same path set/modes/types, 44 identical mode/type/blob entries. Only agents/ALReviewAgent/bcquality.config.yaml differs: exactly one ref substitution from b74967bc5b7a454eae19d6a1250199afd869f064 to 8c9385c16fdb1f21afa6ce957420ead73294d06d; version/every other byte unchanged.

Config blob 54239fecf163a2c90003a0a559f6350826cc8aed → e2e68b42eacbcaa8693e26578a9d9e52bf7a7052; config SHA-256 5ec09c2c57b16340b71d7dffe549ae337417ff374e975da5931cf9d1fe976ad7. Exact parent PR80 checks 2/2 success, BCQuality220 7/7 success at preflight. They do not substitute for the blocked BC-Bench CI gate or a new smoke.

Unchanged V4 fixture/data

The initial PR commit corrected synthetic__http-optional-clean-01: ReadRate(): Decimal consumed an unauthenticated public response body, legitimately triggering security/validate-unauthenticated-response-before-use, not an optional-return false positive. Void CheckRateEndpoint() now does no body reading/parsing/trust/persistence/return. Bare GET/POST remain single-line statements followed by explicit status checks, with fixed endpoints and constant '{}' payload.

• Source remains 31 lines, GET 10, POST 22; expected_comments=[] / ignored_comments=[] preserved.
• Dataset SHA-256 8e8f8afe85bf9a9f3b7fd556a882e1d5eed4c87466ab473181a0c128bb435b93.
• Clean source SHA-256 a0f1b6bc70f5c0dddc0ecc5ffca17b339c73887747b76d56e15438c74c93e2ad.
• Clean patch SHA-256 73f38817423cd93dd708524f275a9adb54379d7644272cde31d536093ed0852e.
• All 148 rows are byte-identical to previous PR941 head 184d58f1. Relative to [Experimental] Second semantic-normalization smoke pin - AB#652973 (no product merge/release) #940, only the original clean entry patch/description differ; the other 147 rows, including the other six selected entries, are unchanged.

Exact entry before original correction / current. Old v2/v3 hashes/authorizations and consumed first-V4 approval are historical only; no pooling or approval reuse.

Validation and exact diff proof

Local exact-head validation: focused 456 passed; full 1244 passed, 2 skipped, 1 deselected; Ruff and all 11 local pre-commit hooks passed, including typing/lockfile checks. Only command-local Windows empty core.fsmonitor propagation was normalized to false; no repo/global config change. AL syntax was not rerun because all dataset/fixture bytes are unchanged. Local success is not a claim that the different remote synthetic merge passed.

Raw-byte proof: the action is exactly one SHA substitution; workflow exactly two inserted selector lines; every other workflow field is equal. Diff/staged checks passed. New normal commit includes required Copilot co-author trailer.

Diff	Paths	SHA-256	Bytes
Incremental from 184d58f1	4	a8e7449cbc2a963af18d51a0d143fab1793863aa8f81857d5ec065fa7b585fae	15,934
Full PR from #940	6	88e06d8b672c5b30e075ee44ceba2f9751cf253a4e3d54a9dc248e5b70d41820	42,756

Incremental: 29 additions/13 deletions; recursive 937 entries, 933 identical. Four paths: action, evaluation workflow, workflow tests, existing docs. Full PR includes the original fixture row and its test: 931 identical entries.

Canonical command: git diff --no-ext-diff --no-textconv --no-color --no-renames --full-index --binary --src-prefix=a/ --dst-prefix=b/ <base> e15e5adbc31754e222b9d2e97057fe9c8c2fb9f6 --. Use base 184d58f19feaadf2b20e7d9f35a10b164f8cf6d4 for incremental or 5cacfe09231352d152d8827c307585e85b15c391 for full PR; hash raw stdout without text re-encoding.

Fresh proposed authorization — NOT GRANTED; CI BLOCKED

New engine/diagnostic preparation over the same V4 data; not a rerun authorization. New explicit approval must bind this head/tree, engine and hashes, and the current CI prerequisite must also be satisfied. Exact ordered matrix:

["synthetic__privacy-008","synthetic__privacy-015","synthetic__security-clean-02","synthetic__errh-tryfunction-swallowed-01","synthetic__http-optional-clean-01","synthetic__http-consumed-false-01","synthetic__http-status-body-01"]

Proposed maxima: 1 dispatch/arm/run, 7 entry generations, 140 reviewer processes, 5 gold-judge invocations; 8 expected comments across 5 judge-eligible entries. Invocation bounds, not provider-call/token/credit/dollar caps. No retry/rerun/replacement/replication/extra adjudicator/publication/PR posting/release/tag/merge. On observed failure stop remaining execution once and audit.

Freeze Bench 0.14.0.dev1, engine 0b01ed8c90f3b07cbc693aa8869f84a16ded338d, BCQuality 8c9385c16fdb1f21afa6ce957420ead73294d06d / 1.6, CLI 1.0.88, root gpt-5.6-sol, leaf gpt-5.6-luna, judge gpt-5.3-codex, serial leaves, configured4/effective1, 30-minute timeout and Medium/Medium. Separately authorized manual dispatch: this branch at exact new head, empty git-ref, matching explicit engine SHA, exact entries, test-run=true, modified-only=false, repeat=1. Preserve existing concurrency/matrix caps, no-tag/no-requeue and summary mock=true. Summary mock does not mock reviewers/judges; no Braintrust/Kusto/leaderboard writes.

Immediately revalidate exact PR/ref/stack/provenance/hash identities, clean checkouts and successful current named-PR CI. Require 7/7 valid terminal reports, zero semantic-consumer failures. Independently inspect generated leaf/root schemas against all source bounds and pinned shared schema; audit raw bytes/hashes, exact ID/range deltas, roles/order/models/CLI/outcomes/usage and executed=declared identity. Optional-clean must be silent across raw/accepted/root/scored output; consumed-false/status-body expected findings recognized. TryFunction remains ordinary additional evidence. Keep complete unmatched/unscored inventory without reinterpretation and reviewer/judge usage separate. NO FURTHER EXECUTION AUTHORIZED.

[github-code-quality]

Code Coverage Overview

Languages: Python

Python / code-coverage/pytest

The overall line coverage in commit e15e5ad in the gggdttt-clean-option... branch is 94%. The line coverage in commit 973eb05 in the gggdttt-prepare-norm... branch is 86%.

Show a line coverage summary of the most impacted files.

File	gggdttt-prepare-norm... 973eb05	gggdttt-clean-option... e15e5ad	+/-
src/bcbench/age...run_manifest.py	0%	95%	+95%
tests/test_mcp_gateway.py	0%	99%	+99%
tests/test_eval...tion_summary.py	0%	99%	+99%
tests/test_codereview.py	0%	100%	+100%
tests/test_cli_commands.py	0%	100%	+100%
tests/test_result_hierarchy.py	0%	100%	+100%
tests/test_pr_review_agent.py	0%	100%	+100%
tests/test_review_workflows.py	0%	100%	+100%
tests/test_pr_r...run_manifest.py	0%	100%	+100%
tests/test_code...iew_fixtures.py	0%	100%	+100%