Forge: complete EP v1.1 consumer foundation

[pcvantol]

Forge consumer/runtime foundation for AUTONOMY_BOOTSTRAP_CLEAN_EP_AND_FIRST_DELIVERY_LOOP.

IMPLEMENTED / QUALIFIED on this branch:

• EP v1.1 request-first HTTP consumer with durable canonical submission/run binding, cold restart recovery, scoped bearer transport and no duplicate ambiguous resend.
• Raw terminal artifact retrieval, SHA-256 before parse, and fail-closed parity for terminal flags, outcomes, delivery qualification, accepted-request identity, provenance and repository evidence.
• Lossless/fail-closed persisted ProducerContract reconstruction, including references and versions; absent legacy contract is distinguished from present malformed data.
• Actual rendered Mission revision is carried into ProducerContract transport metadata; runtime service has bounded wakeable backoff plus canonical cross-process mutation lease qualification.
• Legacy capability-upgrade provenance is revalidated on first and repeated upgrade paths.
• Full local suite and coverage gate: 413 tests, total 89.24%, every Forge component >80.20%.

NOT YET IMPLEMENTED / NOT YET LIVE PROVEN:

• Forge Server implementation and installed qualification: central storage relocation, own HTTP server/API, launchd and installed product integration are deliberately outside Forge: complete EP v1.1 consumer foundation #46. The canonical target requires future OpenAPI/Postman/route-drift qualification.
• EP assurance installed qualification remains EP-owned.
• Dynamic evidence-driven Mission-loop qualification remains separate (docs: define dynamic Living Mission Graph and cross-repo Action DAG #48).
• No live Mission canary or A/B action sequence is claimed.

No EP state, installation or repair loop was changed; no programme grant, activation window, runtime instance or budget was reset.

[pcvantol]

Automated architecture review of 87a1c304a6cfa384e3efd83f6c511b9db25fdeb6 — request/ProducerContract roundtrip (not Human Security Review).

The new _request_document() persistence is useful, but _request() is not yet a lossless, fail-closed restoration of the full contract:

1. ProducerContract.receipt_references, execution_evidence_references, and the persisted contract versions (including nested producer.contract_version) are not restored/validated; construction uses defaults. A non-default contract can therefore change after restart. Please require normalized document equality and contract-digest equality after write/reopen/read, with nonempty reference collections in the test.
2. A PRESENT producer_contract that is not a mapping is treated like an absent legacy contract (contract=None), allowing default reconstruction. Distinguish absent versioned legacy evidence from malformed new-format records; malformed/null/wrong-type contract data must not silently become defaults.
3. The new code uses str(...) coercion extensively. Validate required types/versions and actual prompt/request consistency before construction rather than allowing missing/null values to become strings. Add negative tests for unsupported top-level/nested versions, mismatched prompt content/digest, and malformed metadata. Preserve the defined distinction between source/generation digests and payload-byte digests; do not invent hash semantics.

This commit changes only forge/runtime/runner.py; it adds no regression tests. The reported 11 existing passing tests are a regression baseline, not evidence of the new edge cases. Please add the focused tests in this same PR, then continue with durable submission/run binding, real Mission-revision provenance, original artifact-byte verification, cold-process restart, and service exclusivity/backoff. No new programme grant or budget reset is requested.

The separate server-deployment documentation is on codex/server-deployment-discovery-architecture / Forge PR #47. Preserve independent central Forge storage and product-owned migration; do not introduce an adapter-owned database in a checkout.

[pcvantol]

Automated code/architecture review of exact head 6ec32148180d3a6548aee765a855f2b547a4678b (not a human security approval; no merge, host, grant or deployment mutation).

Progress confirmed: hosted checks are green; durable request/submission/run binding, original artifact-byte hashing, and a valid-hash cross-run rejection test are now present. The previously missing contract reference/version restoration is also present. These should not be reported as wholly unimplemented again.

Blocking consumer correctness findings

1. Terminal outcome/evidence consistency is not fully enforced (forge/scheduler/ep_v11.py). outcome comes from readback.result, but document.run.outcome, document.report.terminal_state, readback.run.state and the readback's delivery_qualified are not reconciled. In an isolated replay of the retrieved mapper function and its evidence dataclasses using the pinned fixtures (NOT a full checkout/host qualification), the baseline passed and a digest-valid wrong-run artifact was correctly rejected. These modified inputs nevertheless returned ExecutionEvidenceOutcome.COMPLETE:

• artifact run.outcome = FAILED while readback still says COMPLETE;
• artifact report.terminal_state = FAILED while readback still says COMPLETE;
• readback result.delivery_qualified = false while artifact carries true;
• accepted_request_digest removed from BOTH artifact/submission and readback/submission (the equality comparison accepts None == None).

Reject contradictory or missing required evidence rather than choosing a convenient field. Validate the versioned artifact/readback schema and explicit presence/type of required fields, outcome/state/qualification parity, and immutable accepted-request identity. Preserve distinct digest semantics: a ProducerContract digest is NOT automatically the EP accepted-request digest. Add negative regressions using the actual adapter plus mapper. This does not assert that the healthy EP producer currently emits these contradictions; it identifies a missing fail-closed consumer check.

2. Present-null ProducerContract still falls back to legacy reconstruction (forge/runtime/runner.py). _request() uses document.get('producer_contract') followed by if contract_document is not None; an explicitly present null is therefore indistinguishable from a genuinely absent legacy field. Reject present-null, and avoid coercing malformed required values into nonempty strings. Add an absent-legacy versus present-null regression and lossless non-default reopen equality/digest evidence. Do not drop the references/version restoration already added.

3. The repeated capability-upgrade path still skips full legacy-provenance validation (forge/operator_identity.py). In authorities == expected, only the new upgrade document's digest is recomputed; the three old grants contribute their stored IDs/digests, then the method returns before the legacy document-validation loop. authorize() verifies the active operator binding, not those grant documents. Validate the original legacy documents, their recomputed digests and identity/binding provenance in BOTH paths. Add upgrade -> reopen -> tampered legacy-provenance rejection in disposable state. Do not recreate the already-recorded live grant or reset budgets.

Integration/qualification claims that need explicit closure

• The HTTP adapter requires mission_revision in ProducerContract.execution_metadata. _release_action() currently uses the default request/contract construction; that default adds intent/repository/workspace metadata but does not itself add the Mission revision. The new HTTP test manually supplies revision 3. Exercise the real Mission materializer -> persisted request -> reopened runner -> HTTP adapter path, and propagate the actual approved Mission revision there; do not satisfy this solely by hand-built test contracts.
• RuntimeServiceLock is a useful advisory lock primitive, but availability to CLI callers is not actual CLI wiring. The current lock test acquires two handles in one process, not a real service process and CLI process. Qualify real process contention and bind lock selection to the resolved runtime, not an unrelated caller path. A per-tick mutation lock does not by itself guarantee a single service process.
• The default wait is time.sleep; keep_running() is checked before, not during, the wait. The current test replaces waiting with list.append and does not prove interruption, the exponential cap, crash recovery or real CLI integration. Distinguish bounded polling shutdown from interruptible waiting, and qualify the promised behavior.

Scope/completion correction

The agreed canonical target (FORGE_SERVER_DEPLOYMENT_TARGET.md on main) remains: an installed headless Forge Server, central SQL/files storage outside Git, state-preserving relocation, own versioned HTTP application API, launchd and authenticated pinned EP instance binding. This PR explicitly says no Forge HTTP server exists yet; a reusable ForgeRuntimeService class is not the installed product. OpenAPI/Postman is a valid API qualification requirement, not grounds to silently defer the entire previously agreed Server.

Keep consumer repair and remaining Server/product integration separately visible. The Server work can be delivered in a coherent follow-up PR if deliberately split, but the programme is NOT complete after #46 and is NOT blocked only on EP. The canary must also prove evidence-driven same-Mission replanning: Forge derives A and then derives/revises successor work from real A evidence, not merely advances a preconfigured A/B list. Cross-repo parallelism, Workspace UI, mDNS and full installer productization remain out of this first canary.

Please close the concrete consumer defects with focused regression evidence before presenting #46 as approved, then complete/qualify the explicit Server and dynamic-Mission integration scope under the applicable existing authorization. No new programme activation, budget reset, EP reinstall or fake evidence is requested.

[pcvantol]

Review-finding update for d289b58:

• ep_v11 now requires terminal/result/run parity, terminal flags, delivery-qualification parity and present, valid, equal accepted-request digests. Adapter-level regressions cover contradictory outcome/report/qualification/flags, missing or invalid accepted digest, unqualified COMPLETE, valid FAILED-without-revision, fixture acceptance and digest-valid wrong-run rejection.
• _request() now distinguishes absent legacy producer_contract from present-null/malformed data, rejects malformed typed values before construction, and has a non-default contract roundtrip/digest regression.
• The default ProducerContract copies an actual rendered prompt mission revision; the Codex materialization regression proves revision 7 reaches restored metadata.
• Legacy capability upgrade now validates all three legacy provenance documents on both first upgrade and idempotent upgraded paths; a disposable-state reopen/tamper regression rejects altered legacy provenance.
• Runtime service uses a wakeable default Event wait; a cross-process flock qualification verifies immediate second-mutator rejection and release after abrupt child termination.

Full local suite and coverage were rerun on this head: total 89.24%, no Forge component below 80.20%. Hosted qualification is now running for this exact head.

[pcvantol]

Follow-up for 6a26c5e: ForgeRuntimeService now accepts an opened canonical RuntimeDatabase and derives its mutation lease from that resolved database path. It no longer accepts a caller-supplied arbitrary path. The service qualification now uses real temporary runtime databases plus separate-process flock contention/release.

[pcvantol]

bab27ea adds the requested durable proof: a non-default ProducerContract is embedded in persisted Mission execution correlation, the runtime database and MissionStateStore are closed and reopened through canonical resolution, and reconstructed contract to_dict() plus digest equal the original.