diff --git a/.ci/behavior-ownership/partition.v1.json b/.ci/behavior-ownership/partition.v1.json index 7b75f7d39..b87e12b13 100644 --- a/.ci/behavior-ownership/partition.v1.json +++ b/.ci/behavior-ownership/partition.v1.json @@ -1496,6 +1496,10 @@ "group": "lifecycle", "target": "backend/app/modules/tasks/post_submit_routing/requests.py" }, + { + "group": "lifecycle", + "target": "backend/app/modules/tasks/post_submit_routing/source.py" + }, { "group": "lifecycle", "target": "backend/app/modules/tasks/queue_router.py" @@ -1665,7 +1669,7 @@ "target": "backend/scripts/validate_test_lane_evidence.py" } ], - "authority_digest": "97f3bee29efd7d0d2214bf11cf1406172feb9375ab157287dc487163a1dfacd2", + "authority_digest": "95fa9e2fa5e30557c54a51940b41141a4ebc67f083345ff66fc359d854525070", "protected_base_commit": "7676ce4347db0c9694962a9b587a20765e16eac6", "schema": "workstream.behavior-ownership-partition.v1" } diff --git a/.commitrail/INDEX.md b/.commitrail/INDEX.md index b11d4b996..cc542838e 100644 --- a/.commitrail/INDEX.md +++ b/.commitrail/INDEX.md @@ -9,7 +9,7 @@ for current product capability. | [WS-DB-002](initiatives/WS-DB-002/OVERVIEW.md) | Complete | Shared UUIDv7 record generation, native-UUID relationships and fresh v0.1 baseline; natural-owner retry custody and aligned CI/local setup | | [WS-MCP-002](initiatives/WS-MCP-002/OVERVIEW.md) | Planned | Nine tools through WS-MCP-002-03: self-service and administrative reads; 18 tools remain and WS-MCP-002-04 administrative grant mutations are next | | [WS-CLI-001](initiatives/WS-CLI-001/OVERVIEW.md) | Planned | Public Go CLI self-service, project inspection, manager browsing and contributor discovery delivered through WS-CLI-001-05; further public journeys and binary distribution remain | -| [WS-ARCH-001](initiatives/WS-ARCH-001/OVERVIEW.md) | Planned | Source storage, inert AUTH contracts, TASK request reservation, hidden exact AUTH preparation and the REV-04C hidden FinalAcceptance/TASK/CON participant are delivered; TASK-before-CHECKERS reservation/current-read custody and ordered admission INSERTs are delivered by 04E1B-B1; hidden routing handlers are next. True admission remains independent of shared acceptance; false activation still requires exact AUTH receipt custody, database complete-set enforcement, currentness race proof and atomic routing activation. | +| [WS-ARCH-001](initiatives/WS-ARCH-001/OVERVIEW.md) | Planned | Source storage, inert AUTH contracts, TASK request reservation, hidden exact AUTH preparation and the REV-04C hidden FinalAcceptance/TASK/CON participant are delivered; TASK-before-CHECKERS reservation/current-read custody and ordered admission INSERTs are delivered by 04E1B-B1; 04E1B-B2 adds exact source preparation without publication; remaining hidden routing handlers are next. True admission remains independent of shared acceptance; false activation still requires exact AUTH receipt custody, database complete-set enforcement, currentness race proof and atomic routing activation. | | [WS-ART-001](initiatives/WS-ART-001/OVERVIEW.md) | Planned | Exact checker input/output custody and packet foundations are delivered; routing integration, remediation and public intake remain. | | [WS-AUTH-001](initiatives/WS-AUTH-001/OVERVIEW.md) | Planned | AUTH-19A commitments, TASK request reservation, ARCH-04E2-A hidden strict PREP matching and the REV-04C hidden FinalAcceptance/TASK/CON participant are delivered; the action remains unavailable, with mandatory exact AUTH receipt input, database closure, audit/outbox and scoped activation still required for the automated path. | | [WS-CON-001](initiatives/WS-CON-001/OVERVIEW.md) | Planned | Contribution/award storage, CON-07 participation and REV-04C hidden FinalAcceptance/TASK/CON composition are delivered; production consumption still requires genuine authority, database complete-set enforcement, currentness race proof, shared audit/outbox, fulfillment-root ordinals and lifecycle activation. | diff --git a/.commitrail/initiatives/WS-ARCH-001/OVERVIEW.md b/.commitrail/initiatives/WS-ARCH-001/OVERVIEW.md index 0f313be0d..4c74b42ad 100644 --- a/.commitrail/initiatives/WS-ARCH-001/OVERVIEW.md +++ b/.commitrail/initiatives/WS-ARCH-001/OVERVIEW.md @@ -1,5 +1,13 @@ # WS-ARCH-001 — Modular monolith boundaries +[ARCH-04E1B-B2](WS-ARCH-001-04E1BB2.md) supplies exact source preparation through +TASK, CHECKERS and historical PROJECTS policy facts. It stages only the existing +routing request; proposed source facts have no fabricated creation timestamp. +Remaining handlers, source publication, current pointers and authority/effect +activation are still required. False composition must acquire its REV lifecycle +fence before TASK and revalidate policy under TASK custody; true admission +remains independent of that fence. + [AUTH-19A](../WS-AUTH-001/WS-AUTH-001-19A.md) delivers inert exact source/receipt contracts and the planned router identity. ARCH-04E1B-A delivers caller-owned routing-request and future source-identity reservation. ARCH-04E2-A now delivers strict hidden diff --git a/.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1BB2.md b/.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1BB2.md new file mode 100644 index 000000000..9eb97216d --- /dev/null +++ b/.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1BB2.md @@ -0,0 +1,125 @@ +# ARCH-04E1B-B2 — Exact routing source preparation + +- Initiative: `WS-ARCH-001` +- Durable disposition: `Complete` +- Intended merge outcome: hidden TASK source preparation joins the existing reserved routing identity to exact current CHECKERS material and historical PROJECTS policy, without publishing a source or applying effects. + +## Intent + +Prepare exact, owner-verified facts for the governed post-submit route. + +## Current behavior + +After 04E1B-B1, reservation and current reads retain TASK-before-CHECKERS custody. +`TaskRoutingRequests.stage` reserves a future manifest identity but receives only +an integer from `require_current_completion`. TASK cannot assemble its source +without reaching into CHECKERS/PROJECTS private tables. The existing PROJECTS +locked-policy port already resolves historical policy bodies and activation +custody. Reuse it; extend the existing CHECKERS completion projection. + +This is the next bounded part of 04E1B-B, not completion of its handlers. + +## Bounded change + +### Allowed + +Allowed files: CHECKERS `api/execution.py`, `execution_coordination.py`; +TASK `api/post_submit_routing.py`, `api/__init__.py`, +`post_submit_routing/requests.py`, `post_submit_routing/source.py`; +`adapters/tasks/__init__.py`; directly affected TASK/CHECKERS contract and +PostgreSQL tests, their existing fixture helpers, lane/ownership registrations; +this record, current ARCH overview/plan/chunk map/parent 04E contract, INDEX, +README, roadmap and canonical checker/TASK specifications where claims change. + +### Not allowed + +Prohibited: migrations, AUTH action activation or synthetic allows, new source +storage or INSERT, routing pointers/status changes, REV/CON effects, outbox +registration/publication, live workers/routes, guide activation changes, +compatibility aliases, retained-data deletion or weakened tests/checks. + +## Design and decisions + +1. Replace the version-only completion return with closed detached verified + completion facts; update all affected callers. CHECKERS retains the same + currentness/phase-receipt/material checks and root transaction requirement. +2. TASK prepares semantic source facts without a fabricated creation timestamp. + The existing persisted manifest facts retain their mandatory database time + and share the same semantic fields. This distinguishes a proposal from a + stored record, not parallel implementations. +3. In the caller root transaction, lock exact TASK/Assignment/Submission scope, + resolve the historical PROJECTS context before acquiring CHECKERS custody, + reserve/recover the existing route identity and construct the exact proposal. + Compare complete activation and Submission lineage, including review boolean, + contribution-policy identity and material facts. No current project policy + can replace locked policy. Return detached facts, never ORM rows. +4. Exact replay reuses the existing reservation. Caller rollback removes newly + staged requests; preparation never commits or creates another owner effect. + +The future false handler must acquire the joint REV lifecycle fence before +TASK, then revalidate the policy under TASK custody. It must not call shared +acceptance for the first time after holding TASK/CHECKERS. True admission stays +independent of the shared acceptance fence. Full handler/receipt/publication +composition remains separately required by 04E1B-B/04E2-B/04E3. + +## Acceptance criteria + +- Real PostgreSQL completed-source fixture returns the exact reserved identity, + Submission/predecessor/contributor, locked policies, checker references/phase + receipts and canonical ART material, with no private storage coordinates. +- Same request replays identically; root rollback removes new reservations. +- Foreign valid source substitutions, stale generation and changed lineage deny; + no source, status, REV/CON, audit or outbox effects are introduced. +- Independent sessions prove preparation retains TASK custody and a successor + that wins first makes the old completion unavailable. +- Managed/raw savepoints remain rejected through the existing root guard. +- False policy remains pure value-contract proof only: guide activation and + genuine false runtime authority remain unavailable. No fake allow or guide + activation is evidence for this chunk. The current hidden intake leaves TASK + in_progress; focused tests explicitly seed the future dispatch-owned + evaluation_pending precondition and separately prove earlier-state denial. + The ART fixture also seeds assignments directly; these tests explicitly set + its missing accepted_at claim precondition after proving that absence denies. + They do not claim authorized claim or initial dispatch proof. + +## Risk and review routing + +Risk: L1. Required focused architecture/reuse, security, QA/test-delta, +CI integrity and documentation review. Human focus: historical policy versus +current policy; currentness lock lifetime; proposal versus persisted source; +no premature runtime/authority claim. The plan review identified absent REV +admission and initial dispatch composition plus the false-handler fence order; +this bounded preparation does not fabricate those dependencies. + +## Evidence + +Run affected pure contracts, real PostgreSQL source preparation/currentness, +module boundaries, Ruff, lane inventory, stale wording, links and Commitrail; +then full hosted CI and exact-head internal review. The four new PostgreSQL preparation cases passed, including exact full-value +comparison, replay/rollback, historical guide preservation, mixed valid source +rejection and retained locks followed by stale-generation rejection. Expanded +root-transaction and cross-project probes and full hosted evidence are tracked +in the PR. +## Reconciliation + +Main reconciled at `31ac857b`; #476 supplies lock repair. +No local spreadsheet export has yet been assumed present. + +Next usable boundary: remaining hidden request/completion handlers, mandatory +AUTH receipt/database/audit/outbox closure, then live composition and remediation. + + +## Review findings + +The final ownership inventory admits only the new source module and rejects +an adjacent activation module. No migration or database schema changed. Local +spreadsheet exports are absent. Plan review retained evaluation_pending and +accepted-assignment guards; older ART-only fixture setup is explicitly arranged +in these new mechanical tests rather than relaxing the production requirements. + +Review corrections: isolate the task-state denial from the missing claim timestamp, +prepare a fully eligible sibling before selector substitution, and prove version-2 +predecessor lookup through a genuinely admitted and evaluated successor ZIP. +The independent-session lock probes establish retained custody; shared B1 race +cases cover the overlapping admission/currentness sequence. Reconcile B2 in the +roadmap trace links and current ARCH dependency tables, not only their summaries. diff --git a/.commitrail/initiatives/WS-ARCH-001/planning/CHUNK_MAP.md b/.commitrail/initiatives/WS-ARCH-001/planning/CHUNK_MAP.md index 2f7768802..b42bffa41 100644 --- a/.commitrail/initiatives/WS-ARCH-001/planning/CHUNK_MAP.md +++ b/.commitrail/initiatives/WS-ARCH-001/planning/CHUNK_MAP.md @@ -1,4 +1,12 @@ # WS-ARCH-001 — Current remaining change map +[ARCH-04E1B-B2](../WS-ARCH-001-04E1BB2.md) supplies exact source preparation through +TASK, CHECKERS and historical PROJECTS policy facts. It stages only the existing +routing request; proposed source facts have no fabricated creation timestamp. +Remaining handlers, source publication, current pointers and authority/effect +activation are still required. False composition must acquire its REV lifecycle +fence before TASK and revalidate policy under TASK custody; true admission +remains independent of that fence. + [AUTH-19A](../../WS-AUTH-001/WS-AUTH-001-19A.md) delivers inert exact source/receipt contracts and the planned router identity. ARCH-04E1B-A delivers caller-owned routing-request and @@ -60,7 +68,7 @@ currentness proof stay within 04E1B-B, before activation and live composition. | [WS-ARCH-001-04D1](../WS-ARCH-001-04D1.md) | Canonical terminal ART material custody | L1 | Complete; valid retained history preserved; invalid upgrades refused | | [WS-ARCH-001-04D2](../WS-ARCH-001-04D2.md) | AUTH exact fixed-service post-submit activation (replaces XINT-06B) | L1 | Complete: exact input, execute and finalize authority; output write/bind remains unavailable | | [WS-ARCH-001-04E1A](../WS-ARCH-001-04E1A.md) | Route-neutral immutable source schema and shared accepted-effects types | L1 | Complete; REV-04C uses a bounded exact-source verifier, with no general routing publication writer/reader, routing authority or current pointer; its hidden effects participant does not make false routing available | -| [WS-ARCH-001-04E](chunks/WS-ARCH-001-04E-canonical-allow-review.md) | TASK current routing: true to canonical `allow_review`, false/pass to shared acceptance | L1 | Both branches: delivered 04E1A/04D2/OUTBOX-02, request reservation 04E1B-A and AUTH preparation 04E2-A -> hidden handlers/currentness proof 04E1B-B -> authority/evidence closure and activation 04E2-B -> live 04E3. False uses delivered REV-04C participation and additionally needs mandatory exact receipt/database/audit closure, fulfillment-root ordinal custody and scoped lifecycle activation; true does not depend on CON. False guide activation also requires 04F remediation | +| [WS-ARCH-001-04E](chunks/WS-ARCH-001-04E-canonical-allow-review.md) | TASK current routing: true to canonical `allow_review`, false/pass to shared acceptance | L1 | Both branches: delivered 04E1A/04D2/OUTBOX-02, request reservation 04E1B-A and AUTH preparation 04E2-A; exact source preparation [04E1B-B2](../WS-ARCH-001-04E1BB2.md) -> remaining handlers/currentness proof 04E1B-B -> authority/evidence closure and activation 04E2-B -> live 04E3. False uses delivered REV-04C participation and additionally needs mandatory exact receipt/database/audit closure, fulfillment-root ordinal custody and scoped lifecycle activation; true does not depend on CON. False guide activation also requires 04F remediation | | [WS-ARCH-001-03D](../WS-ARCH-001-03D.md) | Exact activated historical guide through hidden durable intake; obsolete lookup removed | L1 | Complete; hidden exact post-submit materialization, ARCH-04B2 output custody, ARCH-04C execution, ARCH-04D1/04D2 custody/authority and ARCH-04E1A source-only facts/types delivered; public cutover remains deferred | | [WS-ARCH-001-04F](chunks/WS-ARCH-001-04F-checker-remediation.md) | Contributor-correctable checker failures and same-lineage admission-backed replacement Submission | L1 | Planned after 04E; replaces XINT-05C, required before public 02I, not before REV begins from `allow_review` | diff --git a/.commitrail/initiatives/WS-ARCH-001/planning/PLAN.md b/.commitrail/initiatives/WS-ARCH-001/planning/PLAN.md index abbeef1d2..d98a70627 100644 --- a/.commitrail/initiatives/WS-ARCH-001/planning/PLAN.md +++ b/.commitrail/initiatives/WS-ARCH-001/planning/PLAN.md @@ -1,4 +1,12 @@ # WS-ARCH-001 — Current delivery plan through allow_review +[ARCH-04E1B-B2](../WS-ARCH-001-04E1BB2.md) supplies exact source preparation through +TASK, CHECKERS and historical PROJECTS policy facts. It stages only the existing +routing request; proposed source facts have no fabricated creation timestamp. +Remaining handlers, source publication, current pointers and authority/effect +activation are still required. False composition must acquire its REV lifecycle +fence before TASK and revalidate policy under TASK custody; true admission +remains independent of that fence. + [AUTH-19A](../../WS-AUTH-001/WS-AUTH-001-19A.md) delivers inert exact source/receipt contracts and the planned router identity. ARCH-04E1B-A delivers caller-owned routing-request and @@ -82,6 +90,7 @@ checker-remediation boundary before public Submission cutover. | ARCH-04E1B-A | ARCH-04E1A, AUTH-19A | Complete: TASK caller-session request/source-identity reservation; no source publication, handler or commit | | [ARCH-04E2-A](../WS-ARCH-001-04E2A.md) | ARCH-04E1B-A | Complete: strict AUTH-private resource/request/consequence matcher and nominal fixed-router adapter through canonical PREP; action stays planned/unavailable, with no handle, allow, receipt, source write or effect | | [ARCH-04E1B-B1](../WS-ARCH-001-04E1BB1.md) | REV-04C and existing CHECKERS coordinator | Complete: required TASK reservation/current-read guard, ordered admission INSERTs, exact terminal replay and mechanical race proof; no handler or activation | +| [ARCH-04E1B-B2](../WS-ARCH-001-04E1BB2.md) | 04E1B-B1, existing historical PROJECTS context | Complete: exact detached source preparation and reserved identity; no publication, authority or handler effect | | ARCH-04E1B-B | ARCH-04E2-A and CON-02B; false additionally uses delivered REV-04C shared acceptance participation | Hidden TASK handlers plus TASK-before-CHECKERS currentness and both successor-generation race orders; durable receipt proof follows at exact activation | | Scoped XINT-003-08B controller activation | Existing REV-12A foundation, delivered REV-04C hidden participant and remaining real-writer observation proof | Existing Operator lifecycle-control action for the bounded shared manifest, not human runtime | | ARCH-04E2-B | ARCH-04E1B-B; scoped XINT-003-08B controller activation for false | Require the exact AUTH receipt on the same strict input; add database complete-set and audit/outbox closure; genuine allow commits with all governed effects | diff --git a/.commitrail/initiatives/WS-ARCH-001/planning/chunks/WS-ARCH-001-04E-canonical-allow-review.md b/.commitrail/initiatives/WS-ARCH-001/planning/chunks/WS-ARCH-001-04E-canonical-allow-review.md index 07e91b63c..742f83aa7 100644 --- a/.commitrail/initiatives/WS-ARCH-001/planning/chunks/WS-ARCH-001-04E-canonical-allow-review.md +++ b/.commitrail/initiatives/WS-ARCH-001/planning/chunks/WS-ARCH-001-04E-canonical-allow-review.md @@ -106,7 +106,12 @@ creating human admission, acceptance or contribution effects. [04E1B-B1](../../WS-ARCH-001-04E1BB1.md) installs mandatory TASK-before-CHECKERS reservation custody and exact terminal replay. Mechanical race proof covers acceptance-first reservation rejection and successor-first stale completion - rejection; the full authorized handler races remain required. After 04E1B-A/04E2-A and + rejection. [04E1B-B2](../../WS-ARCH-001-04E1BB2.md) supplies exact + semantic source preparation from current CHECKERS and historical PROJECTS + custody, without source publication or authority. The full authorized + handler races remain required. False composition must take the REV lifecycle + fence before TASK and revalidate the policy after locking; source preparation + does not acquire that fence or authorize later lock-order inversion. After 04E1B-A/04E2-A and CON-02B's handler/claim contract (plus delivered REV-04C acceptance foundations for false), TASK implements unavailable request/event production for its own evaluation-request event and the TASK consumer of diff --git a/README.md b/README.md index 4b9c2abc0..03e6e6220 100644 --- a/README.md +++ b/README.md @@ -186,7 +186,9 @@ no exact AUTH decision-event receipt, and database complete-set enforcement, currentness race proof, shared audit/outbox and lifecycle activation remain. ARCH-04E1B-B1 requires TASK locking before checker reservation, current-result reads and review admission INSERTs, preserving exact read-only reservation replay -after acceptance. Hidden handlers still need complete +after acceptance. ARCH-04E1B-B2 prepares exact source proposals from current +CHECKERS custody and historical PROJECTS policy, without inserting a manifest +or granting authority. Hidden handlers still need complete authorized currentness proof. These prerequisites do not require live human review before the first automated acceptance path. diff --git a/backend/app/adapters/tasks/__init__.py b/backend/app/adapters/tasks/__init__.py index c6fb15bcd..34f0b2706 100644 --- a/backend/app/adapters/tasks/__init__.py +++ b/backend/app/adapters/tasks/__init__.py @@ -188,3 +188,15 @@ def submitted_bundle_port(session: AsyncSession) -> SubmittedBundlePort: """Compose the exact immutable Submission read without private owner imports in ART.""" from app.modules.tasks.submitted_bundle import SubmittedBundleReader return SubmittedBundleReader(session) + + +def routing_source_preparer(session): + """Compose exact hidden source preparation through existing owner ports.""" + from app.adapters.checkers import evaluation_coordinator + from app.adapters.projects import project_locked_policy_context_port + from app.modules.tasks.post_submit_routing.source import TaskRoutingSourcePreparer + + return TaskRoutingSourcePreparer( + session, evaluations=evaluation_coordinator(session), + projects=project_locked_policy_context_port(session), + ) diff --git a/backend/app/modules/checkers/api/execution.py b/backend/app/modules/checkers/api/execution.py index de97cab18..91c30ba55 100644 --- a/backend/app/modules/checkers/api/execution.py +++ b/backend/app/modules/checkers/api/execution.py @@ -232,8 +232,8 @@ class EvaluationCoordinationPort(Protocol): async def require_current_completion( self, event_id: ResourceId, completion: "EvaluationCompletion" - ) -> int: - """Lock exact current allow-review completion; return its stored submission version.""" + ) -> "VerifiedEvaluationCompletion": + """Lock exact current allow-review completion and return retained material facts.""" ... async def reserve_current_evaluation( @@ -260,3 +260,11 @@ class EvaluationCompletion(PostSubmitValue): output_binding_ids: tuple[ResourceId, ...] = Field(max_length=0) execute_evidence_id: ResourceId finalize_evidence_id: ResourceId + + +class VerifiedEvaluationCompletion(PostSubmitValue): + """Current owner-verified completion and retained material, never routing authority.""" + + completion: EvaluationCompletion + submission_version: VersionNumber + material: VerifiedMaterialFacts diff --git a/backend/app/modules/checkers/execution_coordination.py b/backend/app/modules/checkers/execution_coordination.py index b271e0474..956f79167 100644 --- a/backend/app/modules/checkers/execution_coordination.py +++ b/backend/app/modules/checkers/execution_coordination.py @@ -16,6 +16,7 @@ EvaluationCompletion, EvaluationTaskGuard, VerifiedMaterialFacts, + VerifiedEvaluationCompletion, ) from app.modules.checkers.api.post_submit import ( PostSubmissionEvaluationRequest, @@ -188,7 +189,7 @@ async def read_current_result( async def require_current_completion( self, event_id: UUID, completion: EvaluationCompletion - ) -> int: + ) -> VerifiedEvaluationCompletion: """Verify caller completion against retained current custody without granting authority.""" require_transaction(self._session) completion = EvaluationCompletion.model_validate(completion) @@ -235,7 +236,9 @@ async def require_current_completion( or material.byte_count != request.byte_count ): raise CheckerExecutionUnavailable("checker_current_completion_unavailable") - return run.submission_version + return VerifiedEvaluationCompletion( + completion=completion, submission_version=run.submission_version, material=material, + ) class CurrentExecution: diff --git a/backend/app/modules/tasks/api/__init__.py b/backend/app/modules/tasks/api/__init__.py index bd9770311..36c11d323 100644 --- a/backend/app/modules/tasks/api/__init__.py +++ b/backend/app/modules/tasks/api/__init__.py @@ -9,7 +9,8 @@ TaskAcceptedEffectsUnavailable, ) from app.modules.tasks.api.post_submit_routing import ( - TaskPostSubmitManifestFacts, task_post_submit_source_digest, + TaskPostSubmitManifestFacts, TaskPostSubmitSourceProposal, TaskRoutingSourcePreparation, + task_post_submit_source_digest, TaskRoutingSelection, TaskRoutingRequestFacts, task_routing_request_digest, ) @@ -111,7 +112,7 @@ "TaskAcceptedEffectsResult", "TaskAcceptedEffectsUnavailable", "TaskPolicyLineage", - "TaskPostSubmitManifestFacts", + "TaskPostSubmitManifestFacts", "TaskPostSubmitSourceProposal", "TaskRoutingSourcePreparation", "task_post_submit_source_digest", "TaskRoutingSelection", "TaskRoutingRequestFacts", diff --git a/backend/app/modules/tasks/api/post_submit_routing.py b/backend/app/modules/tasks/api/post_submit_routing.py index 26a13f6f0..5d7b73b99 100644 --- a/backend/app/modules/tasks/api/post_submit_routing.py +++ b/backend/app/modules/tasks/api/post_submit_routing.py @@ -22,13 +22,12 @@ _ByteCount = Annotated[StrictInt, Field(ge=0, le=9_223_372_036_854_775_807)] -class TaskPostSubmitManifestFacts(BaseModel): - """Immutable persisted and owner-joined source facts without routing authority.""" +class TaskPostSubmitSourceProposal(BaseModel): + """Detached semantic source proposal; neither stored evidence nor authority.""" model_config = ConfigDict(extra="forbid", frozen=True, strict=True) id: UUID - created_at: AwareDatetime project_id: UUID task_id: UUID submission_id: UUID @@ -87,6 +86,12 @@ def validate_detached_lineage(self) -> Self: return self +class TaskPostSubmitManifestFacts(TaskPostSubmitSourceProposal): + """Persisted source with its mandatory database-owned creation time.""" + + created_at: AwareDatetime + + def task_post_submit_source_digest(source: TaskPostSubmitManifestFacts) -> str: """Commit exact source facts except the database-assigned creation time.""" if type(source) is not TaskPostSubmitManifestFacts: @@ -98,8 +103,9 @@ def task_post_submit_source_digest(source: TaskPostSubmitManifestFacts) -> str: }) -__all__ = ("TaskPostSubmitManifestFacts", "task_post_submit_source_digest", - "TaskRoutingSelection", "TaskRoutingRequestFacts", "task_routing_request_digest") +__all__ = ("TaskPostSubmitSourceProposal", "TaskPostSubmitManifestFacts", "task_post_submit_source_digest", + "TaskRoutingSelection", "TaskRoutingRequestFacts", "task_routing_request_digest", + "TaskRoutingSourcePreparation") class TaskRoutingSelection(BaseModel): @@ -156,3 +162,23 @@ def validate_request_custody(self) -> Self: ): raise ValueError("routing request digest differs") return self + + +class TaskRoutingSourcePreparation(BaseModel): + """Reserved request with matching proposed source; no publication or authority.""" + + model_config = ConfigDict(extra="forbid", frozen=True, strict=True) + + request: TaskRoutingRequestFacts + source: TaskPostSubmitSourceProposal + + @model_validator(mode="after") + def matching_source(self) -> Self: + request = TaskRoutingRequestFacts.model_validate(self.request.model_dump()) + source = TaskPostSubmitSourceProposal.model_validate(self.source.model_dump()) + if request.routing_manifest_id != source.id or request.evaluation_request_digest != source.request_digest: + raise ValueError("routing source identity differs") + for name in TaskRoutingSelection.model_fields: + if name != "evaluation_request_digest" and getattr(request, name) != getattr(source, name): + raise ValueError("routing source selection differs") + return self diff --git a/backend/app/modules/tasks/post_submit_routing/requests.py b/backend/app/modules/tasks/post_submit_routing/requests.py index 46b92b5aa..f6680161d 100644 --- a/backend/app/modules/tasks/post_submit_routing/requests.py +++ b/backend/app/modules/tasks/post_submit_routing/requests.py @@ -7,7 +7,9 @@ from sqlalchemy.ext.asyncio import AsyncSession from app.core.identifiers import new_record_id -from app.modules.checkers.api.execution import EvaluationCompletion, EvaluationCoordinationPort +from app.modules.checkers.api.execution import ( + EvaluationCompletion, EvaluationCoordinationPort, VerifiedEvaluationCompletion, +) from app.modules.tasks.api.post_submit_routing import ( TaskRoutingRequestFacts, TaskRoutingSelection, @@ -25,6 +27,24 @@ class TaskRoutingRequestUnavailable(RuntimeError): """Conceal unavailable or inconsistent request preparation.""" +async def require_routing_transaction(session: AsyncSession) -> None: + """Reject non-root custody before acquiring owner locks.""" + transaction = session.get_transaction() + if transaction is None or not transaction.is_active or session.in_nested_transaction(): + raise TaskRoutingRequestUnavailable("routing_request_caller_transaction_required") + connection = await session.connection() + if connection.in_nested_transaction(): + raise TaskRoutingRequestUnavailable("routing_request_caller_transaction_required") + # PostgreSQL observes native savepoints which SQLAlchemy cannot see. + # Discard the snapshot token; caller transaction completion owns cleanup. + try: + await session.execute(text("SELECT pg_catalog.pg_export_snapshot()")) + except DBAPIError as exc: + if getattr(exc.orig, "sqlstate", None) != "25001": + raise + raise TaskRoutingRequestUnavailable("routing_request_caller_transaction_required") from exc + + class TaskRoutingRequests: """Reserve/recover one request inside a caller-owned transaction; never commit.""" @@ -36,20 +56,14 @@ async def stage( self, event_id: UUID, completion: EvaluationCompletion ) -> TaskRoutingRequestFacts: """Revalidate latest Submission and current completion before reservation or replay.""" - transaction = self._session.get_transaction() - if transaction is None or not transaction.is_active or self._session.in_nested_transaction(): - raise TaskRoutingRequestUnavailable("routing_request_caller_transaction_required") - connection = await self._session.connection() - if connection.in_nested_transaction(): - raise TaskRoutingRequestUnavailable("routing_request_caller_transaction_required") - # PostgreSQL observes native savepoints which SQLAlchemy cannot see. - # Discard the snapshot token; caller transaction completion owns cleanup. - try: - await self._session.execute(text("SELECT pg_catalog.pg_export_snapshot()")) - except DBAPIError as exc: - if getattr(exc.orig, "sqlstate", None) != "25001": - raise - raise TaskRoutingRequestUnavailable("routing_request_caller_transaction_required") from exc + request, _ = await self._stage(event_id, completion) + return request + + async def _stage( + self, event_id: UUID, completion: EvaluationCompletion, + ) -> tuple[TaskRoutingRequestFacts, VerifiedEvaluationCompletion]: + """Share the checked completion with owner-local source preparation.""" + await require_routing_transaction(self._session) completion = EvaluationCompletion.model_validate(completion) if not isinstance(event_id, UUID) or completion.routing_recommendation != "allow_review": raise TaskRoutingRequestUnavailable("routing_request_unavailable") @@ -70,8 +84,8 @@ async def stage( ).limit(1)) if later is not None: raise TaskRoutingRequestUnavailable("routing_request_unavailable") - version = await self._evaluations.require_current_completion(event_id, completion) - if version != submission.version: + verified = await self._evaluations.require_current_completion(event_id, completion) + if verified.submission_version != submission.version: raise TaskRoutingRequestUnavailable("routing_request_unavailable") ref = completion.reference selection = TaskRoutingSelection( @@ -107,4 +121,4 @@ async def stage( or facts.model_dump(include=set(TaskRoutingSelection.model_fields)) != selection.model_dump() ): raise TaskRoutingRequestUnavailable("routing_request_conflict") - return facts + return facts, verified diff --git a/backend/app/modules/tasks/post_submit_routing/source.py b/backend/app/modules/tasks/post_submit_routing/source.py new file mode 100644 index 000000000..b2fab0a22 --- /dev/null +++ b/backend/app/modules/tasks/post_submit_routing/source.py @@ -0,0 +1,138 @@ +"""Prepare exact semantic routing source facts without publishing or authorizing.""" + +import json +from uuid import UUID + +from sqlalchemy import select +from sqlalchemy.ext.asyncio import AsyncSession + +from app.modules.checkers.api.execution import EvaluationCompletion, EvaluationCoordinationPort +from app.modules.projects.api import ( + ProjectLockedPolicyContextPort, ProjectLockedPolicyContextRequest, + ProjectLockedPolicyContextUnavailable, +) +from app.modules.tasks.api.post_submit_routing import ( + TaskPostSubmitSourceProposal, TaskRoutingSourcePreparation, +) +from app.modules.tasks.api.transition_audit import TaskPolicyLineage +from app.modules.tasks.post_submit_routing.requests import ( + TaskRoutingRequests, TaskRoutingRequestUnavailable, require_routing_transaction, +) +from app.modules.tasks.repository import TaskRepository +from app.modules.tasks.models import Submission +from app.modules.tasks.service import TaskService + + +class TaskRoutingSourcePreparer: + """Retain caller locks and stage only the existing routing request identity.""" + + def __init__( + self, session: AsyncSession, *, evaluations: EvaluationCoordinationPort, + projects: ProjectLockedPolicyContextPort, + ) -> None: + self._session = session + self._tasks = TaskRepository(session) + self._requests = TaskRoutingRequests(session, evaluations) + self._projects = projects + + async def prepare( + self, event_id: UUID, completion: EvaluationCompletion, + ) -> TaskRoutingSourcePreparation: + """Resolve TASK then PROJECTS before retaining CHECKERS currentness custody.""" + await require_routing_transaction(self._session) + completion = EvaluationCompletion.model_validate_json(completion.model_dump_json()) + with self._session.no_autoflush: + return await self._prepare(event_id, completion) + + async def _prepare(self, event_id, completion): + task = await self._tasks.lock_project_task(completion.project_id, completion.task_id) + if task is None or task.status != "evaluation_pending": + raise TaskRoutingRequestUnavailable("routing_source_unavailable") + # Read selectors without taking a Submission lock ahead of Assignment. + submission = await self._tasks.get_latest_submission_for_task(str(completion.task_id)) + if submission is None or submission.id != str(completion.submission_id): + raise TaskRoutingRequestUnavailable("routing_source_unavailable") + assignment = await self._tasks.lock_accepted_assignment( + project_id=completion.project_id, task_id=completion.task_id, + assignment_id=UUID(submission.task_assignment_id), + ) + submission = await self._tasks.get_latest_submission_for_task( + str(completion.task_id), for_update=True, populate_existing=True, + ) + if assignment is None or submission is None or not ( + submission.id == str(completion.submission_id) + and submission.task_assignment_id == assignment.id + and assignment.status == "active" and assignment.released_at is None + and assignment.accepted_at is not None + and submission.contributor_id == assignment.contributor_id == task.assigned_to + and submission.contribution_policy_version_id + == assignment.submitter_contribution_policy_version_id + == task.locked_contribution_policy_version_id + ): + raise TaskRoutingRequestUnavailable("routing_source_unavailable") + try: + context = await self._projects.lock_locked_policy_context(ProjectLockedPolicyContextRequest( + project_id=completion.project_id, guide_version=submission.locked_guide_version, + source_snapshot_id=UUID(submission.locked_guide_source_snapshot_id), + source_snapshot_hash=submission.locked_guide_source_snapshot_hash, + effective_policy_id=UUID(submission.locked_effective_project_submission_artifact_policy_id), + effective_policy_hash=submission.locked_effective_project_submission_artifact_policy_hash, + pre_submit_policy_id=UUID(submission.locked_pre_submit_checker_policy_id), + pre_submit_policy_bundle_hash=submission.locked_pre_submit_checker_bundle_hash, + )) + stamps = TaskService._policy_stamps(context) + if any(getattr(task, key) != value for key, value in stamps.items()): + raise ValueError("task policy differs") + fields = {} + for key in TaskPolicyLineage.model_fields: + value = getattr(submission, key, None) + if key == "locked_contribution_policy_version_id": + value = submission.contribution_policy_version_id + if key.endswith("_id"): + value = UUID(str(value)) + fields[key] = value + lineage = TaskPolicyLineage(**fields) + for key in TaskPolicyLineage.model_fields: + value = getattr(task, key) + if key.endswith("_id"): + value = UUID(str(value)) + if value != getattr(lineage, key): + raise ValueError("submission policy differs") + human_review_required = json.loads(context.review_policy.value)["human_review_required"] + except (ProjectLockedPolicyContextUnavailable, ValueError, TypeError, KeyError) as exc: + raise TaskRoutingRequestUnavailable("routing_source_unavailable") from exc + request, verified = await self._requests._stage(event_id, completion) + material = verified.material + if not ( + material.admission_id == UUID(submission.submission_bundle_admission_id) + and material.binding_id == UUID(submission.artifact_binding_id) + and material.content_id == UUID(submission.artifact_content_id) + ): + raise TaskRoutingRequestUnavailable("routing_source_unavailable") + predecessor = None + if submission.supersedes_submission_id: + predecessor = await self._session.scalar(select(Submission).where( + Submission.id == submission.supersedes_submission_id, + Submission.task_id == submission.task_id, + )) + if predecessor is None or predecessor.task_id != submission.task_id: + raise TaskRoutingRequestUnavailable("routing_source_unavailable") + source = TaskPostSubmitSourceProposal( + id=request.routing_manifest_id, project_id=completion.project_id, + task_id=completion.task_id, submission_id=completion.submission_id, + submission_version=verified.submission_version, assignment_id=UUID(assignment.id), + contributor_id=UUID(submission.contributor_id), + contribution_policy_version_id=submission.contribution_policy_version_id, + checker_run_id=request.checker_run_id, evaluation_request_id=request.evaluation_request_id, + request_digest=request.evaluation_request_digest, + evaluation_generation=request.evaluation_generation, result_id=request.result_id, + result_digest=request.result_digest, completion_event_id=event_id, + execute_evidence_id=verified.completion.execute_evidence_id, + finalize_evidence_id=verified.completion.finalize_evidence_id, + human_review_required=human_review_required, + predecessor_submission_id=UUID(predecessor.id) if predecessor else None, + predecessor_submission_version=predecessor.version if predecessor else None, + locked_policy=lineage, routing_recommendation="allow_review", + **material.model_dump(exclude={"submission_id", "submission_version"}), + ) + return TaskRoutingSourcePreparation(request=request, source=source) diff --git a/backend/scripts/behavior_ownership.py b/backend/scripts/behavior_ownership.py index 3463f0757..70987f744 100644 --- a/backend/scripts/behavior_ownership.py +++ b/backend/scripts/behavior_ownership.py @@ -174,6 +174,7 @@ REV_04B_ACCEPTANCE_TARGETS = frozenset({"backend/app/modules/reviews/acceptance/models.py", "backend/app/modules/reviews/acceptance/schemas.py"}) REV_04A_SOURCE_TARGETS = frozenset({"backend/app/modules/reviews/decision/models.py", "backend/app/modules/reviews/decision/schemas.py"}) REV_03B_PACKET_TARGETS = frozenset({'backend/app/modules/reviews/packet/models.py', 'backend/app/modules/reviews/packet/repository.py', 'backend/app/modules/reviews/packet/schemas.py'}) +ARCH_04E1BB2_SOURCE_TARGETS = frozenset({"backend/app/modules/tasks/post_submit_routing/source.py"}) ARCH_04E1BB1_GUARD_TARGETS = frozenset({"backend/app/modules/tasks/post_submit_routing/evaluation_guard.py"}) ARCH_04E1BA_REQUEST_TARGETS = frozenset({"backend/app/modules/tasks/post_submit_routing/requests.py"}) OBSERVABILITY_FOUNDATION_TARGETS = frozenset( @@ -791,6 +792,7 @@ def _validate_additive_partition_transition( | REV_03B_PACKET_TARGETS | ARCH_04E1BA_REQUEST_TARGETS | ARCH_04E1BB1_GUARD_TARGETS + | ARCH_04E1BB2_SOURCE_TARGETS | OBSERVABILITY_FOUNDATION_TARGETS | ARCH_04E1A_SOURCE_TARGETS | ARCH_04D2_AUTHORITY_TARGETS diff --git a/backend/scripts/test_lane_catalogue.py b/backend/scripts/test_lane_catalogue.py index 226e4eca4..fc6a1b76e 100644 --- a/backend/scripts/test_lane_catalogue.py +++ b/backend/scripts/test_lane_catalogue.py @@ -257,6 +257,7 @@ class TestLane: "tests/tasks/post_submit_routing/test_evaluation_guard.py", "tests/tasks/post_submit_routing/test_evaluation_currentness.py", "tests/tasks/post_submit_routing/test_review_admission_currentness.py", + "tests/tasks/post_submit_routing/test_source_preparation.py", "tests/tasks/accepted_effects/test_contracts.py", "tests/tasks/accepted_effects/test_postgresql.py", # Frozen policy participation shares measured project capacity, not the full schema lane. diff --git a/backend/tests/tasks/post_submit_routing/support.py b/backend/tests/tasks/post_submit_routing/support.py index 785ab25b6..639e89d0f 100644 --- a/backend/tests/tasks/post_submit_routing/support.py +++ b/backend/tests/tasks/post_submit_routing/support.py @@ -601,14 +601,58 @@ async def successor_submission(h): await session.execute(text("UPDATE public.workstream_tasks SET status='needs_revision' WHERE id=:id"), {"id": h.request.task_id}) context = _context(actor_profile_id=as_uuid(original.contributor_id), identity_link_id=as_uuid(identity_link_id)) + successor = SimpleNamespace(**(vars(h) | {"data": revision_archive(h.data)})) + return await _create_submission(successor, context, h.request.task_id, h.request.assignment_id, h.request.submission_id) + + +def revision_archive(data): + """Change one real ZIP member while preserving the governed packet.""" from io import BytesIO from zipfile import ZipFile revised = BytesIO() - with ZipFile(BytesIO(h.data)) as source, ZipFile(revised, "w") as target: + with ZipFile(BytesIO(data)) as source, ZipFile(revised, "w") as target: for item in source.infolist(): - data = source.read(item) + content = source.read(item) if item.filename == "notes.txt": - data += b"\nRevision: corrected the implementation.\n" - target.writestr(item, data) - successor = SimpleNamespace(**(vars(h) | {"data": revised.getvalue()})) - return await _create_submission(successor, context, h.request.task_id, h.request.assignment_id, h.request.submission_id) + content += b"\nRevision: corrected the implementation.\n" + target.writestr(item, content) + return revised.getvalue() + + +async def completed_successor_source(h): + """Evaluate a real successor created through verified ZIP admission.""" + from app.modules.checkers.api import PostSubmitManifestEntry + from tests.post_submit_materialization_helpers import _archive_facts + + created = await successor_submission(h) + data = revision_archive(h.data) + _, manifest, _, digest = _archive_facts(data) + async with h.factory() as session: + facts = await submitted_bundle_port(session).read(SubmittedBundleRequest( + h.request.project_id, h.request.task_id, created.submission_id, + )) + context = asdict(facts.context) + request = change_request( + h.request, evaluation_request_id=new_record_id(), evaluation_generation=1, + submission_id=created.submission_id, submission_version=created.submission_version, + content_id=created.artifact_content_id, binding_id=created.artifact_binding_id, + content_sha256=digest, byte_count=len(data), + expected_context=ExpectedPostSubmitContext(**context), + structural_input=h.request.structural_input.model_copy(update={ + "package_hash": digest, + "observed_context": ObservedPostSubmitContext(**context), + "manifest": tuple(PostSubmitManifestEntry( + artifact=e.normalized_path, hash=e.sha256, size_bytes=e.byte_count, + ) for e in manifest.entries if e.sha256 is not None), + }), + ) + successor = SimpleNamespace(**(vars(h) | {"data": data, "request": request, "source": {}})) + await rebuild_real_request(successor) + await reserve(successor) + successor.result = await live_executor(successor).evaluate_post_submission(successor.request) + async with h.factory() as session: + run = await session.get(CheckerRun, str(successor.result.attempt_id)) + assert run.status == "completed" and run.routing_recommendation == "allow_review" + successor.material = dict(run.material_custody) + successor.source = await source_values(successor) + return successor diff --git a/backend/tests/tasks/post_submit_routing/test_contracts.py b/backend/tests/tasks/post_submit_routing/test_contracts.py index 374af14ef..0e7f5c7fd 100644 --- a/backend/tests/tasks/post_submit_routing/test_contracts.py +++ b/backend/tests/tasks/post_submit_routing/test_contracts.py @@ -162,8 +162,9 @@ def test_false_source_value_is_transport_only() -> None: def test_routing_source_foundation_has_no_runtime_entry() -> None: assert post_submit_routing.__all__ == ( - "TaskPostSubmitManifestFacts", "task_post_submit_source_digest", + "TaskPostSubmitSourceProposal", "TaskPostSubmitManifestFacts", "task_post_submit_source_digest", "TaskRoutingSelection", "TaskRoutingRequestFacts", "task_routing_request_digest", + "TaskRoutingSourcePreparation", ) for name in post_submit_routing.__all__: assert getattr(task_api, name) is getattr(post_submit_routing, name) @@ -172,7 +173,7 @@ def test_routing_source_foundation_has_no_runtime_entry() -> None: name for name, value in vars(post_submit_routing).items() if isclass(value) and value.__module__ == post_submit_routing.__name__ - } == {"TaskPostSubmitManifestFacts", "TaskRoutingSelection", "TaskRoutingRequestFacts"} + } == {"TaskPostSubmitSourceProposal", "TaskPostSubmitManifestFacts", "TaskRoutingSelection", "TaskRoutingRequestFacts", "TaskRoutingSourcePreparation"} assert not hasattr(task_api, "TaskRoutingRequests") action = ActionId.TASK_POST_SUBMIT_ROUTE assert ACTION_BY_ID[action].permission_id is PermissionId.TASK_POST_SUBMIT_ROUTE @@ -180,3 +181,17 @@ def test_routing_source_foundation_has_no_runtime_entry() -> None: assert SERVICE_ACTIONS_BY_IDENTITY[ServiceIdentity.TASK_POST_SUBMIT_ROUTER] == {action} with pytest.raises(ValueError, match="authorization action is not active"): resolve_executable_action(action) + + +@pytest.mark.parametrize("human_review_required", [True, False]) +def test_proposal_has_no_persisted_timestamp(human_review_required): + values = _source_values(human_review_required=human_review_required) + source = post_submit_routing.TaskPostSubmitSourceProposal( + **{key: value for key, value in values.items() if key != "created_at"} + ) + assert source.human_review_required is human_review_required + assert "created_at" not in source.model_dump() + with pytest.raises(ValidationError, match="Extra inputs"): + post_submit_routing.TaskPostSubmitSourceProposal(**values) + with pytest.raises(ValidationError, match="created_at"): + TaskPostSubmitManifestFacts(**source.model_dump()) diff --git a/backend/tests/tasks/post_submit_routing/test_requests.py b/backend/tests/tasks/post_submit_routing/test_requests.py index b454aa9aa..15dad85e7 100644 --- a/backend/tests/tasks/post_submit_routing/test_requests.py +++ b/backend/tests/tasks/post_submit_routing/test_requests.py @@ -143,7 +143,8 @@ async def test_reservation_holds_currentness_lock(tmp_path, isolated_database_en await other.execute(text(f"SELECT 1 FROM public.{table} WHERE {key}=:id FOR UPDATE NOWAIT"), {"id": value}) -async def test_cross_project_cannot_lock_foreign_task(tmp_path, isolated_database_env): +@pytest.mark.parametrize("entry", ["request", "source"]) +async def test_cross_project_cannot_lock_foreign_task(tmp_path, isolated_database_env, entry): async with completed_source(tmp_path / "one", isolated_database_env) as first: async with completed_source(tmp_path / "two", isolated_database_env, provision_services=False, storage_settings=first.settings) as second: assert first.request.project_id != second.request.project_id @@ -153,8 +154,13 @@ async def test_cross_project_cannot_lock_foreign_task(tmp_path, isolated_databas async with first.factory() as intruder, intruder.begin(): await intruder.execute(text("SET LOCAL lock_timeout = '250ms'")) # A SQL timeout is deliberately not accepted as concealed denial. - with pytest.raises(TaskRoutingRequestUnavailable, match="routing_request_unavailable"): - await stage(intruder, first, mixed) + message = "routing_request_unavailable" if entry == "request" else "routing_source_unavailable" + with pytest.raises(TaskRoutingRequestUnavailable, match=message): + if entry == "request": + await stage(intruder, first, mixed) + else: + from app.adapters.tasks import routing_source_preparer + await routing_source_preparer(intruder).prepare(first.source["completion_event_id"], mixed) async def test_replay_rejects_older_still_submitted_version(tmp_path, isolated_database_env): @@ -187,17 +193,22 @@ def untrusted_completion(): ) +@pytest.mark.parametrize("entry", ["request", "source"]) @pytest.mark.parametrize("kind", [ "missing", "session_nested", "raw_session", "raw_connection", "raw_driver", "external_connection_nested", "external_create_savepoint", ]) -async def test_request_requires_database_root_transaction(isolated_database_env, kind): +async def test_request_requires_database_root_transaction(isolated_database_env, kind, entry): from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine engine = create_async_engine(isolated_database_env) event, completion = untrusted_completion() async def rejected(session): with pytest.raises(TaskRoutingRequestUnavailable, match="routing_request_caller_transaction_required"): - await TaskRoutingRequests(session, evaluation_coordinator(session)).stage(event, completion) + if entry == "request": + await TaskRoutingRequests(session, evaluation_coordinator(session)).stage(event, completion) + else: + from app.adapters.tasks import routing_source_preparer + await routing_source_preparer(session).prepare(event, completion) try: if kind.startswith("external_"): async with engine.connect() as connection: diff --git a/backend/tests/tasks/post_submit_routing/test_review_admission_currentness.py b/backend/tests/tasks/post_submit_routing/test_review_admission_currentness.py index b3f49be7b..43c5a428d 100644 --- a/backend/tests/tasks/post_submit_routing/test_review_admission_currentness.py +++ b/backend/tests/tasks/post_submit_routing/test_review_admission_currentness.py @@ -40,9 +40,12 @@ async def _read(session, h, reader): result = await coordinator.read_current_result(h.request) assert result.result == h.result else: - assert await coordinator.require_current_completion( + verified = await coordinator.require_current_completion( h.source["completion_event_id"], completion_for(h), - ) == h.request.submission_version + ) + assert verified.submission_version == h.request.submission_version + assert verified.completion == completion_for(h) + assert verified.material.model_dump(mode="json") == h.material async def _admit(session, queue, admission): diff --git a/backend/tests/tasks/post_submit_routing/test_source_preparation.py b/backend/tests/tasks/post_submit_routing/test_source_preparation.py new file mode 100644 index 000000000..511c396b3 --- /dev/null +++ b/backend/tests/tasks/post_submit_routing/test_source_preparation.py @@ -0,0 +1,163 @@ +"""Exact owner-composed routing proposals without publication or authority.""" + +from contextlib import asynccontextmanager + +import pytest +from sqlalchemy import func, select, text +from sqlalchemy.exc import DBAPIError + +from app.adapters.tasks import routing_source_preparer +from app.modules.checkers.api.execution import CheckerExecutionUnavailable +from app.modules.tasks.post_submit_routing.models import TaskRoutingRequest +from app.modules.tasks.post_submit_routing.requests import TaskRoutingRequestUnavailable +from tests.checkers.execution.support import reserve +from .support import ( + completed_source, completed_sibling_source, completion_for, next_request, + joined_source_facts, source_count, activate_successor_guide, completed_successor_source, +) +from .test_requests import effect_snapshot + + +@asynccontextmanager +async def routing_source(tmp_path, database_url): + """Real completed evaluation with future dispatch state explicitly seeded. + + This proves mechanical preparation, not live dispatch or routing authority. + """ + async with completed_source(tmp_path, database_url) as h: + async with h.factory() as session, session.begin(): + # Establish the claim precondition first: the status denial must not + # succeed because accepted_at is also absent. + started = await session.execute(text( + "UPDATE public.task_assignments SET accepted_at=clock_timestamp() " + "WHERE id=:id AND accepted_at IS NULL" + ), {"id": h.request.assignment_id}) + assert started.rowcount == 1 + before = await effect_snapshot(session) + with pytest.raises(TaskRoutingRequestUnavailable, match="routing_source_unavailable"): + await prepare(session, h) + assert await effect_snapshot(session) == before + assert await session.scalar(select(func.count()).select_from(TaskRoutingRequest)) == 0 + changed = await session.execute(text( + "UPDATE public.workstream_tasks SET status='evaluation_pending' " + "WHERE id=:id AND status='in_progress'" + ), {"id": h.request.task_id}) + assert changed.rowcount == 1 + # Isolate missing accepted_at under an otherwise eligible state. + await session.execute(text( + "UPDATE public.task_assignments SET accepted_at=NULL WHERE id=:id" + ), {"id": h.request.assignment_id}) + with pytest.raises(TaskRoutingRequestUnavailable, match="routing_source_unavailable"): + await prepare(session, h) + await session.execute(text( + "UPDATE public.task_assignments SET accepted_at=clock_timestamp() WHERE id=:id" + ), {"id": h.request.assignment_id}) + yield h + + +async def prepare(session, h, completion=None): + return await routing_source_preparer(session).prepare( + h.source["completion_event_id"], completion or completion_for(h), + ) + + +async def test_exact_proposal_replay_and_rollback(tmp_path, isolated_database_env): + from datetime import UTC, datetime + async with routing_source(tmp_path, isolated_database_env) as h: + expected = await joined_source_facts(h, h.source | {"created_at": datetime.now(UTC)}) + async with h.factory() as session: + await session.begin() + before = await effect_snapshot(session) + result = await prepare(session, h) + assert result.source.model_dump() == expected.model_dump(exclude={"created_at"}) | { + "id": result.request.routing_manifest_id, + } + assert "created_at" not in result.source.model_dump() + assert await source_count(session) == 0 + assert await effect_snapshot(session) == before + await session.rollback() + async with h.factory() as session, session.begin(): + assert await session.scalar(select(func.count()).select_from(TaskRoutingRequest)) == 0 + committed = await prepare(session, h) + async with h.factory() as session, session.begin(): + assert await prepare(session, h) == committed + assert await effect_snapshot(session) == before + assert await source_count(session) == 0 + assert await session.scalar(select(func.count()).select_from(TaskRoutingRequest)) == 1 + + +async def test_historical_policy_survives_new_guide(tmp_path, isolated_database_env): + async with routing_source(tmp_path, isolated_database_env) as h: + async with h.factory() as session, session.begin(): + before = await prepare(session, h) + await activate_successor_guide(h) + async with h.factory() as session, session.begin(): + assert await prepare(session, h) == before + + +async def test_mixed_stored_sources_reject_without_effects(tmp_path, isolated_database_env): + async with routing_source(tmp_path, isolated_database_env) as h: + sibling = await completed_sibling_source(h) + async with h.factory() as session, session.begin(): + await session.execute(text("UPDATE public.workstream_tasks SET status='evaluation_pending' WHERE id=:id"), {"id": sibling.request.task_id}) + await session.execute(text("UPDATE public.task_assignments SET accepted_at=clock_timestamp() WHERE id=:id"), {"id": sibling.request.assignment_id}) + async with h.factory() as session: + await session.begin() + # A complete sibling is a positive control before mixing identities. + await prepare(session, sibling) + await session.rollback() + original, other = completion_for(h), completion_for(sibling) + for field in ("task_id", "submission_id", "reference", "execute_evidence_id", "finalize_evidence_id"): + async with h.factory() as session, session.begin(): + before = await effect_snapshot(session) + with pytest.raises((TaskRoutingRequestUnavailable, CheckerExecutionUnavailable)): + await prepare(session, h, original.model_copy(update={field: getattr(other, field)})) + assert await effect_snapshot(session) == before + assert await source_count(session) == 0 + assert await session.scalar(select(func.count()).select_from(TaskRoutingRequest)) == 0 + + +async def test_preparation_retains_parent_custody_and_successor_wins_next(tmp_path, isolated_database_env): + async with routing_source(tmp_path, isolated_database_env) as h: + old = completion_for(h) + async with h.factory() as first, first.begin(): + await prepare(first, h) + for table, key, value in ( + ("workstream_tasks", "id", h.request.task_id), + ("projects", "id", h.request.project_id), + ("task_assignments", "id", h.request.assignment_id), + ("submissions", "id", h.request.submission_id), + ("checker_submission_fences", "submission_id", h.request.submission_id), + ): + async with h.factory() as other: + with pytest.raises(DBAPIError) as failure: + await other.execute(text(f"SELECT 1 FROM public.{table} WHERE {key}=:id FOR UPDATE NOWAIT"), {"id": value}) + assert failure.value.orig.sqlstate == "55P03" + await next_request(h) + await reserve(h) + async with h.factory() as session, session.begin(): + with pytest.raises(CheckerExecutionUnavailable, match="current_request_unavailable"): + await prepare(session, h, old) + assert await source_count(session) == 0 + + +async def test_successor_preparation_matches_stored_predecessor(tmp_path, isolated_database_env): + from datetime import UTC, datetime + async with routing_source(tmp_path, isolated_database_env) as original: + successor = await completed_successor_source(original) + expected = await joined_source_facts(successor, successor.source | {"created_at": datetime.now(UTC)}) + async with original.factory() as session, session.begin(): + changed = await session.execute(text( + "UPDATE public.workstream_tasks SET status='evaluation_pending' " + "WHERE id=:id AND status='needs_revision'" + ), {"id": original.request.task_id}) + assert changed.rowcount == 1 + async with original.factory() as session, session.begin(): + before = await effect_snapshot(session) + prepared = await prepare(session, successor) + assert prepared.source.model_dump() == expected.model_dump(exclude={"created_at"}) | {"id": prepared.request.routing_manifest_id} + assert prepared.source.submission_version == 2 + assert prepared.source.predecessor_submission_id == original.request.submission_id + assert prepared.source.predecessor_submission_version == 1 + assert await source_count(session) == 0 + assert await effect_snapshot(session) == before diff --git a/backend/tests/test_behavior_ownership.py b/backend/tests/test_behavior_ownership.py index 3dee646ca..9b880a8bb 100644 --- a/backend/tests/test_behavior_ownership.py +++ b/backend/tests/test_behavior_ownership.py @@ -2360,9 +2360,13 @@ def test_acceptance_contract_relocation_requires_public_replacement() -> None: ownership._validate_additive_partition_transition(_partition([retained]), trusted) -def test_evaluation_guard_ownership_does_not_allow_routing_activation(): - target = "backend/app/modules/tasks/post_submit_routing/evaluation_guard.py" - assert ownership.ARCH_04E1BB1_GUARD_TARGETS == {target} +@pytest.mark.parametrize(("filename", "targets"), [ + ("evaluation_guard.py", ownership.ARCH_04E1BB1_GUARD_TARGETS), + ("source.py", ownership.ARCH_04E1BB2_SOURCE_TARGETS), +]) +def test_evaluation_guard_ownership_does_not_allow_routing_activation(filename, targets): + target = "backend/app/modules/tasks/post_submit_routing/" + filename + assert targets == {target} assert ownership.group_for_target(target) == "lifecycle" retained = "backend/app/core/config.py" trusted = _partition([retained]) diff --git a/backend/tests/test_ci_lane_catalogue.py b/backend/tests/test_ci_lane_catalogue.py index 45c9bcd9a..1a8edc42a 100644 --- a/backend/tests/test_ci_lane_catalogue.py +++ b/backend/tests/test_ci_lane_catalogue.py @@ -57,6 +57,7 @@ def test_measured_hotspots_have_explicit_semantic_owners() -> None: "tests/tasks/post_submit_routing/test_evaluation_guard.py", "tests/tasks/post_submit_routing/test_evaluation_currentness.py", "tests/tasks/post_submit_routing/test_review_admission_currentness.py", + "tests/tasks/post_submit_routing/test_source_preparation.py", "tests/tasks/accepted_effects/test_contracts.py", "tests/tasks/accepted_effects/test_postgresql.py", "tests/contributions/records/test_contracts.py", @@ -803,6 +804,7 @@ def test_evaluation_custody_proofs_use_project_lane_headroom(): "tests/tasks/post_submit_routing/test_evaluation_guard.py", "tests/tasks/post_submit_routing/test_evaluation_currentness.py", "tests/tasks/post_submit_routing/test_review_admission_currentness.py", + "tests/tasks/post_submit_routing/test_source_preparation.py", } assert expected <= set(catalogue.PROJECT_MODULES) assert all(catalogue.PARTITION_LANES_BY_MODULE[path] == catalogue.PARTITIONED_PROJECT_LANES for path in expected) diff --git a/docs/roadmap_status.md b/docs/roadmap_status.md index 611d651cf..f282b110e 100644 --- a/docs/roadmap_status.md +++ b/docs/roadmap_status.md @@ -153,7 +153,10 @@ release work; diagnostics do not establish lifecycle truth. [ARCH-04E1B-B1](../.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1BB1.md) adds required TASK-before-CHECKERS reservation/current-result locking, ordered -review admission INSERTs, exact terminal replay and mechanical race controls. Hidden routing handlers and their +review admission INSERTs, exact terminal replay and mechanical race controls. +[ARCH-04E1B-B2](../.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1BB2.md) +adds exact hidden source preparation through existing owner ports, without +source publication or authority. Remaining routing handlers and their complete authorized transaction proof remain next. ## Pre-Submission And Post-Submission Checking @@ -201,7 +204,7 @@ cannot be reused as post-submission review-gate evidence. See the | Contributor artifact preparation | **Hidden and proven** | One outer ZIP; bounded scratch inspection; canonical manifest; platform and project prechecks; unchanged-work rejection; durable put intent; verification; capacity-charged ready admission; hidden final handoff validates the exact activated historical guide through owner ports | Complete the later public admission-only cutover | | Pre-submission intake checking | **Hidden with approved-guide lineage** | Separate versioned pre-submission catalogue, locked effective-plan compilation, platform/project checks during continuous preparation, blocking feedback before Submission creation, and one internal phase command covering execution/replay with the JSON precheck removed; ARCH-03D connects approved-guide lineage through the final durable handoff | Complete the canonical public cutover after evaluation/remediation prerequisites; passing intake must never substitute for post-submit evaluation | | Immutable Submission creation | **Hidden foundation; public packet creation retired** | Contributor preparation authority; durable pre-submit reservation and exact completed-evidence recovery without rerunning checks; atomic admission consumption; TASK-owned admission-backed creation with exact assignment ContributionPolicyVersion and locked policy lineage; fixed-service artifact binding; replay/concurrency/rollback proof | Finish downstream evaluation and the canonical public integration. The retained submission-list GET is not a usable creation POST | -| Post-submission evaluation and `allow_review` | **Hidden exact preparation; routing planned** | One canonical CHECKER post-submit catalogue/compiler used by existing consumers, internal phase service with exact fixed-service post-submit authority, hidden value contracts and structural-handler conformance; ARCH-04B/04B2 input and output custody; ARCH-04C durable execution and current-result custody; ARCH-04D1 canonical ART material custody; ARCH-04D2 exact phase authority and receipts; ARCH-04E1A immutable route-neutral source table and detached facts; AUTH-19A inert source/receipt commitments and planned router identity; ARCH-04E1B-A immutable request/source-identity reservation and exact replay; ARCH-04E2-A strict preparation; REV-04C hidden FinalAcceptance/TASK/CON participant; ARCH-04E1B-B1 required TASK reservation/current-read guard, ordered admission INSERTs and mechanical race controls | Hidden handlers 04E1B-B are next. True routing remains independent of shared acceptance. False activation at 04E2-B requires the same strict participant input to carry the mandatory exact AUTH decision-event receipt, database-enforced FinalAcceptance/TASK/CON completeness, shared audit/outbox and scoped lifecycle/fulfillment custody. 04E1B-B1 supplies reservation/current-read and admission INSERT lock ordering; the remaining handlers must prove complete authorized routing/currentness race orders. No handler, current pointer, route, currentness guarantee or acceptance effect is live; 04F remediation still precedes false guide activation | +| Post-submission evaluation and `allow_review` | **Hidden exact preparation; routing planned** | One canonical CHECKER post-submit catalogue/compiler used by existing consumers, internal phase service with exact fixed-service post-submit authority, hidden value contracts and structural-handler conformance; ARCH-04B/04B2 input and output custody; ARCH-04C durable execution and current-result custody; ARCH-04D1 canonical ART material custody; ARCH-04D2 exact phase authority and receipts; ARCH-04E1A immutable route-neutral source table and detached facts; AUTH-19A inert source/receipt commitments and planned router identity; ARCH-04E1B-A immutable request/source-identity reservation and exact replay; ARCH-04E2-A strict preparation; REV-04C hidden FinalAcceptance/TASK/CON participant; ARCH-04E1B-B1 required TASK reservation/current-read guard, ordered admission INSERTs and mechanical race controls; ARCH-04E1B-B2 exact source preparation without publication | Remaining handlers 04E1B-B are next. True routing remains independent of shared acceptance. False activation at 04E2-B requires the same strict participant input to carry the mandatory exact AUTH decision-event receipt, database-enforced FinalAcceptance/TASK/CON completeness, shared audit/outbox and scoped lifecycle/fulfillment custody. 04E1B-B1 supplies reservation/current-read and admission INSERT lock ordering; the remaining handlers must prove complete authorized routing/currentness race orders. No handler, current pointer, route, currentness guarantee or acceptance effect is live; 04F remediation still precedes false guide activation | | Review queue and lease | **Hidden persistence foundation** | Queue/admission idempotency and ReviewLease/preference persistence; complete unavailable REV action/principal catalogue and typed AUTH contracts; ART-07A1 metadata-only packet contract and REV-03B immutable normalized packet persistence with live ingest custody; REV-04A Review/finding/resolution and completed request storage | Future resolver proof; canonical admission from `allow_review`; claim/lease/packet authority; lease copies the Submission-stamped policy version with no CON lookup | | Review decision and revision | **Hidden acceptance core; runtime planned** | Review/revision policy identities and mutation authority; REV-04A immutable Review storage; REV-04B source storage; REV-04C hidden source-neutral FinalAcceptance/TASK/CON participant; approved same-task revision-rebase semantics | Add the mandatory exact AUTH receipt to the same participant input and complete database/audit/outbox closure before either trigger consumes it; then authorize atomic human decisions, revision preparation, replay and recovery | | Contribution and compensation truth | **Hidden shared participant plus public policy administration** | ContributionPolicyVersion persistence; lifecycle-audit participant; adapter bindings; public Finance policy administration; REV-04B shared acceptance source storage; CON-03C immutable ContributionRecord/CompensationAward storage; CON-07 source-neutral submitter participant and complete award sets; REV-04C composes it with FinalAcceptance/TASK effects in the caller transaction | Add authority/evidence and complete-set database closure before production consumption. Only actual Reviews create reviewer records. Add audit/outbox and fulfillment-root/ordinal custody before activation; no public recognition or fulfillment route is live | @@ -548,7 +551,7 @@ The next dependency-safe product sequence is: exactly replay the contribution and complete zero/one/two frozen award set; paid replay checks its correlation and unpaid replay retains none. REV-04C now composes that result with FinalAcceptance and TASK effects. Hidden handlers - 04E1B-B are next. Delivered 04E1B-B1 makes TASK-before-CHECKERS reservation, + 04E1B-B remain after delivered 04E1B-B2 exact source preparation. Delivered 04E1B-B1 makes TASK-before-CHECKERS reservation, current-read and admission INSERT ordering mandatory, including intermediate admission waits and both mechanical race controls; the full authorized acceptance/successor race proof remains required. At activation 04E2-B make the @@ -676,6 +679,7 @@ Delivered foundations (not a claim of full public integration) CON-07 hidden source-neutral submitter participant + complete frozen award set REV-04C hidden FinalAcceptance + TASK effects + CON composition (no authority/route) ARCH-04E1B-B1 ordered reservation/current reads + admission INSERTs (no activation) + ARCH-04E1B-B2 exact source proposal (no publication or authority) Remaining integration both branches: handlers 04E1B-B -> activation 04E2-B -> live 04E3 dispatch/routing @@ -816,6 +820,7 @@ remaining trace sequence is: and [ARCH-04D2 exact service authority](../.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04D2.md) lead to delivered [ARCH-04E1A source facts](../.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1A.md). + [ARCH-04E1B-B2](../.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1BB2.md) adds exact source preparation without publication or authority. Delivered [ART-07A1 packet types](../.commitrail/initiatives/WS-ART-001/WS-ART-001-07A1.md), [REV-03B packet persistence](../.commitrail/initiatives/WS-REV-001/WS-REV-001-03B.md), [REV-04A Review source storage](../.commitrail/initiatives/WS-REV-001/WS-REV-001-04A.md) and [REV-04B acceptance storage](../.commitrail/initiatives/WS-REV-001/WS-REV-001-04B.md), diff --git a/docs/spec_chunk_4_task_queue_assignment.md b/docs/spec_chunk_4_task_queue_assignment.md index b54eabf0c..2be7786f3 100644 --- a/docs/spec_chunk_4_task_queue_assignment.md +++ b/docs/spec_chunk_4_task_queue_assignment.md @@ -541,8 +541,8 @@ ARCH-04E1B-A provides caller-owned request preparation for an exact current completed `allow_review` evaluation. TASK locks the project-qualified Task and its latest submitted Submission before CHECKERS locks its fence and run. The CHECKERS public coordination port verifies the supplied completion against -retained result, event, phase receipts and material, returning the stored -Submission version. Neither completion values nor request facts grant authority. +retained result, event, phase receipts and material, returning closed verified +completion facts with the stored Submission version and material custody. Neither completion values nor request facts grant authority. `task_post_submit_routing_requests` reserves distinct generated routing-operation and future manifest UUIDv7 identities. Its canonical digest binds the exact owner, @@ -556,3 +556,14 @@ publication, TASK transition, Review, FinalAcceptance or ContributionRecord is created. ARCH-04E2-A delivers strict hidden AUTH preparation and receipt-staging contracts; routing remains unavailable. Later atomic publication must bind the manifest to the reserved identity and verify genuine immutable AUTH evidence. + + +ARCH-04E1B-B2 adds hidden exact source preparation. It requires the task's +`evaluation_pending` state and active accepted assignment, resolves historical +PROJECTS policy after TASK locks and before CHECKERS custody, and combines the +verified completion with the existing reservation. A proposal contains semantic +source facts only; the persisted manifest additionally requires its database +creation time. No source INSERT or authority follows from constructing a proposal. +The current hidden intake does not yet set `evaluation_pending`; the later atomic +dispatch composition owns that transition. Mechanical preparation tests explicitly +arrange that state and the claim timestamp, without claiming live dispatch/claim.