From a8d65e4c900e817cd7a1813107d87b0ea2aa4955 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 04:24:53 +0100 Subject: [PATCH 01/20] Add artifact admission and prepared put attempts --- .agent-loop/LOOP_STATE.md | 31 +- .agent-loop/WORK_QUEUE.md | 11 +- .../CHUNK_MAP.md | 4 +- .../STATUS.md | 44 +- ...1-02C1-admission-put-attempt-foundation.md | 4 +- .../merge-intents/WS-ART-001-02C1.json | 9 + .github/workflows/backend.yml | 8 + .../versions/0027_artifact_admission.py | 384 ++++++ backend/app/core/config.py | 14 + backend/app/modules/artifacts/models.py | 248 ++++ backend/app/modules/artifacts/repository.py | 280 ++++- backend/app/modules/artifacts/schemas.py | 61 + backend/app/modules/artifacts/service.py | 725 +++++++---- backend/tests/artifact_store_helpers.py | 12 + backend/tests/test_artifact_admission.py | 895 +++++++++++++ backend/tests/test_artifact_architecture.py | 25 +- backend/tests/test_artifact_cleanup_wiring.py | 5 + backend/tests/test_artifacts.py | 1109 +---------------- .../tests/test_aws_credential_isolation.py | 2 + backend/tests/test_config.py | 35 + backend/tests/test_s3_artifact_store.py | 6 +- docs/spec_artifact_storage_service.md | 29 +- scripts/test_agent_gates.py | 29 +- 23 files changed, 2598 insertions(+), 1372 deletions(-) create mode 100644 .agent-loop/merge-intents/WS-ART-001-02C1.json create mode 100644 backend/alembic/versions/0027_artifact_admission.py create mode 100644 backend/app/modules/artifacts/schemas.py create mode 100644 backend/tests/test_artifact_admission.py diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index 1cc333487..43403e3ec 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -20,24 +20,27 @@ `eeb3dc2` recorded its two administrative reads and stopped. - PR #141 merged `WS-ART-001-02A3` into `main` as `a10d901` on 2026-07-18. PR #151 then merged `WS-ART-001-02B1` as `1b5422f` on 2026-07-19; - ART-02C1 remains inactive pending signed memory and a separate explicit start. + the user then explicitly started ART-02C1. +- Active ART implementation chunk: `WS-ART-001-02C1` on + `codex/ws-art-001-02c1-admission-put-attempt`. +- The ART worktree consumes merged AUTH, REV, and CON contracts without + editing or activating their independently owned runtime behavior. - AUTH-09D-A merged through PR #148 as `99ae4c9`; signed schema-v2 memory at `cf8a3e8` recorded the stopped gate and exact 09D-B successor. - AUTH-09D-B merged through PR #152 as `93dd392`; signed schema-v2 memory at `912a6254` stopped and named the contributor foundation as its exact successor. +- PR #153 merged `WS-AUTH-001-CONTRIBUTOR-FOUNDATION` as `8d5eb15`; ART-02C1 + now integrates that trusted `main` state without activating AUTH-09E. - PR #119 merged `WS-AUTH-001-05B` as `ad71c7e`. - PR #120 merged `WS-ART-001-OBJECT-STORAGE-AMENDMENT` as `4408256`. - PR #122 merged the first automated post-merge memory implementation as `fc89fb6`; its schema-v1 cross-initiative next pointer is superseded by the schema-v2 initiative-local clean cut. -- Active implementation chunk: `WS-AUTH-001-CONTRIBUTOR-FOUNDATION`, explicitly - started by the user on 2026-07-19 from trusted `main` at `93dd392`. Current - exact contract `2a21166d` passed required L1 preimplementation review; - initial findings are repaired, and exact code SHA `4d1fc507` passed all nine - required internal tracks. PR publication and external checks are the current - gate; Backend must still prove 78/90 percent aggregate coverage. It changes no action - availability, and no service caller becomes executable before AUTH-09E. +- Current ART gate: repair the Backend migration-head assertions found by PR + #154 CI, rerun deterministic proof and all nine exact-SHA internal reviewer + tracks, then return to external checks and explicit human review. No later + ART chunk starts automatically. - Scope checkpoint: AWS S3 is the only v0.1 production provider; MinIO is local/CI S3 protocol proof; LocalStorage is focused development/test; R2 and Flow Node are deferred. Product modules receive narrow artifact capabilities, @@ -55,13 +58,11 @@ admission, prepared mutation authority, and exact AUTH-only activation chunks; neither reconciliation PR activates feature behavior. - Parallel artifact checkpoint: ART-02A1, ART-02A2, ART-02A3, and ART-02B1 - merged through PRs #127, #129, #141, and #151. ART-02B1 adds real MinIO - protocol proof plus a fail-closed, runtime-ineligible native AWS profile; - ART-02C1 remains inactive. -- Authorization checkpoint: AUTH-07B through AUTH-09D-B merged through PRs - #130, #131, #132, #143, #146, #148, and #152. The contributor foundation is - internally approved at code SHA `4d1fc507`; AUTH-09E remains - inactive. + merged through PRs #127, #129, #141, and #151. ART-02C1 is active and adds + only durable admission plus prepared put-attempt state before provider I/O. +- Authorization checkpoint: AUTH-07B through AUTH-09D-B and the contributor + foundation merged through PRs #130, #131, #132, #143, #146, #148, #152, + and #153. AUTH-09E remains inactive. - Parallel coverage work: `WS-QUAL-001-01B2` remains paused. Its last official whole-app result is `6466/8159` statements (`79.249908%`); no replacement evidence exists. diff --git a/.agent-loop/WORK_QUEUE.md b/.agent-loop/WORK_QUEUE.md index 429b977c2..89d1efb03 100644 --- a/.agent-loop/WORK_QUEUE.md +++ b/.agent-loop/WORK_QUEUE.md @@ -4,7 +4,7 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| -| `WS-AUTH-001-CONTRIBUTOR-FOUNDATION` | Contributor Fields And Canonical-Human Lineage | L1 | Internal review passed at `4d1fc507`; PR/external checks pending; aggregate coverage mandatory in Backend | +| `WS-ART-001-02C1` | Admission And Put-Attempt Foundation | L1 | PR #154 open; current-main integration and Backend migration-head CI repair in progress | Live post-merge state remains read from signed `automation/loop-memory` output. This authored queue records the separately approved parallel chunks. @@ -17,7 +17,6 @@ output. This authored queue records the separately approved parallel chunks. | `WS-AUTH-001-09E` | Fixed Service Runtime Admission | L1 | Inactive until contributor-foundation merge/memory and explicit user start | | `WS-QUAL-001-02` | Project Service Coverage | L1 | Inactive until 01B2 merge/memory plus explicit user start | | `WS-POL-002-04` | Locked Runtime Execution And Routing Hardening | L1 | Inactive pending relevant authorization proof and a separate explicit user start | -| `WS-ART-001-02C1` | Admission And Put-Attempt Foundation | L1 | Inactive until signed 02B1 merge memory and explicit user start | | `WS-ART-001-02C2` | Verification Publication And Fencing | L1 | Inactive until 02C1 merge and explicit user start | | `WS-ART-001-02C3` | Recovery Attempt And Idempotency Chain | L1 | Inactive until 02C2 merge and explicit user start | | `WS-ART-001-02D` | Operator Artifact Operations And AWS Readiness | L1 | Inactive until 02C3 and exact AUTH prerequisites | @@ -26,7 +25,7 @@ output. This authored queue records the separately approved parallel chunks. | Chunk | Title | Risk | Status | |---|---|---:|---| -| `WS-ART-001-02B1` | S3-Compatible MinIO And AWS | L1 | Merged through PR #151 as `1b5422f` on 2026-07-19 | +| `WS-ART-001-02B1` | S3-Compatible MinIO And AWS | L1 | Merged through PR #151 as `1b5422fc` on 2026-07-19 | | `WS-AUTH-001-09D-A` | Profile Lifecycle And Evidence Repair | L1 | Merged through PR #148 as `99ae4c9` on 2026-07-18 | | `WS-AUTH-001-09C` | Actor And Identity-Link Administration Reads | L1 | Merged through PR #146 as `0ffdabf` on 2026-07-18 | | `WS-ENG-001-01` | Codex-native zero-trust loop bootstrap | L1 | Merged through PR #23 on 2026-06-20 | @@ -105,9 +104,9 @@ Coverage R10 merged through PR #108. Do not start 01B2, chunk 02, or another coverage implementation chunk from this worktree. `WS-ART-001-01`, the AWS-first planning amendment, `02A1`, `02A2`, `02A3`, and -`02B1` are merged; PR #151 merged `02B1` as `1b5422f`. R2 and Flow Node are -deferred. `02C1` remains inactive until signed merge memory and a separate -explicit start. +`02B1` are merged; PR #151 merged `02B1` as `1b5422fc`. R2 and Flow Node are +deferred. The user explicitly started `02C1` on 2026-07-19. `02C2` remains +inactive until `02C1` merges and receives a separate explicit start. Coverage work proceeds independently in its own worktree and is not owned by this AUTH queue update. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md index 087bd357b..dd01eba89 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md @@ -10,8 +10,8 @@ Each chunk is one PR. No later chunk starts automatically. | `WS-ART-001-02A1` | Install only ADR 0014's small typed external-service adapter/factory foundation without migrating a capability. | L1 | Merged through PR #127 as `f64a8e5` | | `WS-ART-001-02A2` | Add bounded committed-source preparation and inactive scratch-cleanup mechanics without changing the active v1 port. | L1 | Merged through PR #129 as `9a04434` on 2026-07-16 | | `WS-ART-001-02A3` | Replace ArtifactStore v1 with byte-only v2, activate API-startup and Celery Beat scratch cleanup, migrate schema/callers/factory, and remove `flow_node` in one atomic clean cut. | L1 | Merged through PR #141 as `a10d901` on 2026-07-18 | -| `WS-ART-001-02B1` | Implement the S3-compatible adapter, MinIO integration, and AWS S3 production profile. | L1 | Active after 02A3 merged through PR #141 and explicit user start | -| `WS-ART-001-02C1` | Add the generic durable-byte admission ledger and durable put-attempt state foundation without provider execution. | L1 | Proposed after 02B1 | +| `WS-ART-001-02B1` | Implement the S3-compatible adapter, MinIO integration, and AWS S3 production profile. | L1 | Merged through PR #151 as `1b5422fc` on 2026-07-19 | +| `WS-ART-001-02C1` | Add the generic durable-byte admission ledger and durable put-attempt state foundation without provider execution. | L1 | Active after PR #151 and explicit user start on 2026-07-19 | | `WS-ART-001-02C2` | Add put resolution, verification publication, complete-object observation, immutable receipts, and PostgreSQL execution fencing without recovery attempts or routes. | L1 | Proposed after 02C1 | | `WS-ART-001-02C3` | Add the recovery-attempt model and exact idempotent source-job to retry-job chain without public or Operator routes. | L1 | Proposed after 02C2 | | `WS-ART-001-02D` | Add hidden Operator content/job/retry/recovery/audit APIs, canonical resource composition, and production-readiness checks while actions and provider profiles remain inactive. | L1 | Proposed after 02C3, AUTH-09E, and `WS-AUTH-001-ART-CUSTODY` | diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md index af1a1509b..fec9fb02f 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md @@ -1,4 +1,4 @@ -# Status: WS-ART-001 S3-Compatible Object Storage Amendment +# Status: WS-ART-001 Immutable Artifact Storage ## Current State @@ -6,9 +6,10 @@ Original planning merged through PR #97, artifact/LocalStorage foundation merged through PR #101, the AWS-first object-storage amendment merged through PR #120 as `4408256`, the external-service adapter foundation merged through PR #127 as `f64a8e5`, committed-source preparation merged through PR #129 as -`9a04434`, and the ArtifactStore v2 Local clean cut merged through PR #141 as -`a10d901` on 2026-07-18. The user explicitly started `WS-ART-001-02B1` on -2026-07-18. +`9a04434`, the ArtifactStore v2 Local clean cut merged through PR #141 as +`a10d901`, and S3-compatible MinIO/AWS preparation merged through PR #151 as +`1b5422fc` on 2026-07-19. The user explicitly started `WS-ART-001-02C1` on +2026-07-19. The planning-only cross-initiative boundary reconciliation merged through PR #139 as `5d353b6`, and AUTH's owner reconciliation merged through PR #140 as @@ -34,29 +35,26 @@ approval or reusable evidence. Its source remains on branch ## Current Work -`WS-ART-001-02B1` is active. It adds one `S3CompatibleArtifactStore`, runs the -shared ArtifactStore v2 vectors against real digest-pinned MinIO, and validates -an isolated native-AWS workload-identity profile. MinIO is runtime-eligible -only in local/development/test after the PostgreSQL namespace claim. Native AWS -remains runtime-ineligible and fails with -`artifact_provider_live_proof_required` before factory construction, -credential resolution, namespace claim, or provider I/O. No product ingest, -durable admission, put-attempt resolution, verification job, recovery route, -or optional-provider runtime is activated. R2 and Flow Node remain deferred. +`WS-ART-001-02C1` is active. It adds the PostgreSQL durable-byte admission +ledger, closed internal guide/contributor/checker-output requests, and one +`prepared` `ArtifactPutAttempt` created atomically before provider I/O. Scope +limits are explicit configuration; callers cannot supply scope collections; +exact content is charged once per task, producer, project, and deployment +scope. Provider execution, verification, publication, recovery, routes, and +product cutover remain inactive. Native AWS remains runtime-ineligible. R2 and +Flow Node remain deferred. ## Next Proposed Chunk -`02C1` owns generic durable-byte admission and put-attempt state only after -`02B1` merges and receives a separate explicit start. Neither R2 nor Flow Node +`02C2` may add fenced put resolution and verification publication only after +`02C1` merges and receives a separate explicit start. Neither R2 nor Flow Node has a v0.1 chunk. ## Gate -The reviewed implementation recorded: "The current gate is deterministic 02B1 -proof followed by all nine exact-SHA internal reviewer tracks." After integrating -latest `main`, including merged REV planning, that gate passed again at -`9cd5620e` after addressing all four valid CodeRabbit findings, with no ART -runtime or ownership drift. The remaining gate is the post-fix GitHub Actions -and CodeRabbit rerun plus explicit human review. Durable admission, put attempts, -verification publication, and recovery remain in later owning chunks. No later -artifact chunk starts automatically, and only the user may approve merge. +The current gate is deterministic 02C1 proof followed by all nine exact-SHA +internal reviewer tracks. GitHub Actions, CodeRabbit, and explicit human review +follow only after internal evidence is complete and every reviewer session is +closed. Provider execution, verification publication, and recovery remain in +later owning chunks. No later artifact chunk starts automatically, and only the +user may approve merge. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C1-admission-put-attempt-foundation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C1-admission-put-attempt-foundation.md index 75b25d2ec..310580a9e 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C1-admission-put-attempt-foundation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C1-admission-put-attempt-foundation.md @@ -1,6 +1,6 @@ # Chunk Contract: WS-ART-001-02C1 - Admission And Put-Attempt Foundation -Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 02B1 +Initiative: `WS-ART-001` | Risk: L1 | Status: Active after explicit user start Artifact contract phase: `artifact_store_cutover` @@ -80,7 +80,7 @@ coverage report --include='app/modules/audit/*' --precision=2 --fail-under=90 ```bash docker compose up -d --wait postgres redis minio -(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_artifact_admission.py tests/test_config.py -q --cov=app.interfaces.artifact_operations --cov=app.modules.artifacts --cov=app.modules.audit --cov=app.core.config --cov-report=term-missing --cov-fail-under=90) +(cd backend && WORKSTREAM_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_artifact_admission.py tests/test_artifact_architecture.py tests/test_artifact_cleanup_wiring.py tests/test_artifact_preparation.py tests/test_artifact_store_conformance.py tests/test_artifacts.py tests/test_local_artifact_store.py tests/test_s3_artifact_store.py tests/test_audit.py tests/test_config.py -q --cov=app.interfaces.artifact_operations --cov=app.modules.artifacts --cov=app.modules.audit --cov=app.core.config --cov-report=term-missing --cov-fail-under=90) (metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) (cd backend && .venv/bin/ruff check app tests) python3 scripts/check_stale_artifact_contracts.py diff --git a/.agent-loop/merge-intents/WS-ART-001-02C1.json b/.agent-loop/merge-intents/WS-ART-001-02C1.json new file mode 100644 index 000000000..2b1ebeb24 --- /dev/null +++ b/.agent-loop/merge-intents/WS-ART-001-02C1.json @@ -0,0 +1,9 @@ +{ + "chunk_id": "WS-ART-001-02C1", + "chunk_title": "Admission And Put-Attempt Foundation", + "initiative_id": "WS-ART-001", + "next_chunk_id": "WS-ART-001-02C2", + "next_chunk_title": "Verification Publication And Fencing", + "next_requires_explicit_start": true, + "schema_version": 2 +} diff --git a/.github/workflows/backend.yml b/.github/workflows/backend.yml index 56e887446..b3101fb87 100644 --- a/.github/workflows/backend.yml +++ b/.github/workflows/backend.yml @@ -142,6 +142,14 @@ jobs: --precision=2 --fail-under=90 + - name: Audit subsystem coverage + working-directory: backend + run: >- + coverage report + --include='app/modules/audit/*' + --precision=2 + --fail-under=90 + - name: Actor subsystem coverage working-directory: backend run: >- diff --git a/backend/alembic/versions/0027_artifact_admission.py b/backend/alembic/versions/0027_artifact_admission.py new file mode 100644 index 000000000..e01604434 --- /dev/null +++ b/backend/alembic/versions/0027_artifact_admission.py @@ -0,0 +1,384 @@ +"""add durable-byte admission and prepared put attempts + +Revision ID: 0027_artifact_admission +Revises: 0026_actor_profile_lifecycle +Create Date: 2026-07-19 +""" + +from __future__ import annotations + +from alembic import op +import sqlalchemy as sa + + +revision = "0027_artifact_admission" +down_revision = "0026_actor_profile_lifecycle" +branch_labels = depends_on = None + +_ADMISSION_TABLES = ( + "artifact_put_attempt_charges", + "artifact_put_attempts", + "artifact_admission_charges", + "artifact_admission_scopes", +) + + +def _refuse_populated_admission_downgrade() -> None: + """Refuse to destroy durable admission, charge, or attempt evidence.""" + connection = op.get_bind() + connection.execute( + sa.text( + "lock table " + + ", ".join(_ADMISSION_TABLES) + + " in access exclusive mode" + ) + ) + if any( + connection.execute( + sa.text(f"select exists(select 1 from {table_name})") + ).scalar() + for table_name in _ADMISSION_TABLES + ): + raise RuntimeError("cannot downgrade populated artifact admission ledger") + + +def upgrade() -> None: + """Install generic admission and pre-I/O attempt state.""" + op.create_unique_constraint( + "uq_artifact_storage_namespace_id_fingerprint", + "artifact_storage_namespaces", + ["id", "namespace_fingerprint"], + ) + + op.create_table( + "artifact_admission_scopes", + sa.Column("scope_type", sa.String(20), nullable=False), + sa.Column("scope_id", sa.String(120), nullable=False), + sa.Column("limit_bytes", sa.BigInteger(), nullable=False), + sa.Column("counted_bytes", sa.BigInteger(), nullable=False, server_default="0"), + sa.Column("cas_version", sa.BigInteger(), nullable=False, server_default="0"), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column( + "updated_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "scope_type in ('deployment', 'project', 'producer', 'task')", + name="scope_type", + ), + sa.CheckConstraint( + "octet_length(scope_id) between 1 and 120", + name="scope_id_bounds", + ), + sa.CheckConstraint("limit_bytes > 0", name="limit_positive"), + sa.CheckConstraint( + "counted_bytes >= 0 and counted_bytes <= limit_bytes", + name="counted_bytes_within_limit", + ), + sa.CheckConstraint("cas_version >= 0", name="cas_nonnegative"), + sa.PrimaryKeyConstraint("scope_type", "scope_id"), + ) + + op.create_table( + "artifact_admission_charges", + sa.Column("id", sa.String(36), nullable=False), + sa.Column("scope_type", sa.String(20), nullable=False), + sa.Column("scope_id", sa.String(120), nullable=False), + sa.Column("sha256", sa.String(71), nullable=False), + sa.Column("byte_count", sa.BigInteger(), nullable=False), + sa.Column("producer_type", sa.String(30), nullable=False), + sa.Column("producer_ref", sa.String(120), nullable=False), + sa.Column("creating_operation_identity", sa.String(71), nullable=False), + sa.Column("state", sa.String(20), nullable=False, server_default="provisional"), + sa.Column("cas_version", sa.BigInteger(), nullable=False, server_default="0"), + sa.Column( + "reserved_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("released_at", sa.DateTime(timezone=True), nullable=True), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column( + "updated_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "sha256 ~ '^sha256:[0-9a-f]{64}$'", + name="sha256_shape", + ), + sa.CheckConstraint("byte_count >= 0", name="byte_count_nonnegative"), + sa.CheckConstraint( + "producer_type in ('actor_profile', 'service_identity')", + name="producer_type", + ), + sa.CheckConstraint( + "creating_operation_identity ~ '^sha256:[0-9a-f]{64}$'", + name="operation_identity_shape", + ), + sa.CheckConstraint( + "state in ('provisional', 'completed', 'released')", + name="state", + ), + sa.CheckConstraint("cas_version >= 0", name="cas_nonnegative"), + sa.CheckConstraint( + "(state = 'completed') = (completed_at is not null)", + name="completed_timestamp", + ), + sa.CheckConstraint( + "state != 'released' or released_at is not null", + name="released_timestamp", + ), + sa.ForeignKeyConstraint( + ["scope_type", "scope_id"], + [ + "artifact_admission_scopes.scope_type", + "artifact_admission_scopes.scope_id", + ], + name="fk_artifact_admission_charges_scope", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "scope_type", + "scope_id", + "sha256", + "byte_count", + name="uq_artifact_admission_charge_scope_content", + ), + ) + + op.create_table( + "artifact_put_attempts", + sa.Column("id", sa.String(36), nullable=False), + sa.Column("producer_request_type", sa.String(30), nullable=False), + sa.Column("producer_type", sa.String(30), nullable=False), + sa.Column("producer_ref", sa.String(120), nullable=False), + sa.Column("project_id", sa.String(36), nullable=False), + sa.Column("task_id", sa.String(36), nullable=True), + sa.Column("guide_source_item_id", sa.String(36), nullable=True), + sa.Column("upload_item_id", sa.String(36), nullable=True), + sa.Column("checker_run_id", sa.String(36), nullable=True), + sa.Column("logical_role", sa.String(100), nullable=True), + sa.Column("sha256", sa.String(71), nullable=False), + sa.Column("byte_count", sa.BigInteger(), nullable=False), + sa.Column("media_type", sa.String(255), nullable=False), + sa.Column("storage_namespace_id", sa.String(20), nullable=False), + sa.Column("namespace_fingerprint", sa.String(71), nullable=False), + sa.Column("canonical_target", sa.String(1024), nullable=False), + sa.Column("operation_identity", sa.String(71), nullable=False), + sa.Column("request_digest", sa.String(71), nullable=False), + sa.Column("status", sa.String(40), nullable=False, server_default="prepared"), + sa.Column( + "next_run_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column("executor_id", sa.String(36), nullable=True), + sa.Column("lease_expires_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("execution_generation", sa.BigInteger(), nullable=False, server_default="0"), + sa.Column("terminal_result_code", sa.String(100), nullable=True), + sa.Column("replica_id", sa.String(36), nullable=True), + sa.Column("receipt_id", sa.String(36), nullable=True), + sa.Column("cas_version", sa.BigInteger(), nullable=False, server_default="0"), + sa.Column( + "prepared_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column("terminal_at", sa.DateTime(timezone=True), nullable=True), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column( + "updated_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "producer_request_type in ('guide', 'contributor', 'checker_output')", + name="producer_request_type", + ), + sa.CheckConstraint( + "producer_type in ('actor_profile', 'service_identity')", + name="producer_type", + ), + sa.CheckConstraint( + "((producer_request_type in ('guide', 'contributor') " + "and producer_type = 'actor_profile' and " + "producer_ref ~ '^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-" + "[89ab][0-9a-f]{3}-[0-9a-f]{12}$') or " + "(producer_request_type = 'checker_output' " + "and producer_type = 'service_identity' " + "and producer_ref = 'workstream.artifact.checker_output'))", + name="producer_identity", + ), + sa.CheckConstraint("sha256 ~ '^sha256:[0-9a-f]{64}$'", name="sha256_shape"), + sa.CheckConstraint("byte_count >= 0", name="byte_count_nonnegative"), + sa.CheckConstraint( + "canonical_target ~ '^sha256/[0-9a-f]{2}/[0-9a-f]{62}$'", + name="canonical_target_shape", + ), + sa.CheckConstraint( + "operation_identity ~ '^sha256:[0-9a-f]{64}$'", + name="operation_identity_shape", + ), + sa.CheckConstraint( + "request_digest ~ '^sha256:[0-9a-f]{64}$'", + name="request_digest_shape", + ), + sa.CheckConstraint( + "status in ('prepared', 'put_in_flight', 'acknowledgement_unknown', " + "'object_confirmed', 'absent_replay_required', 'integrity_mismatch', " + "'provider_unavailable', 'conflict')", + name="status", + ), + sa.CheckConstraint( + "(executor_id is null) = (lease_expires_at is null)", + name="executor_lease_pair", + ), + sa.CheckConstraint( + "execution_generation >= 0 and cas_version >= 0", + name="versions_nonnegative", + ), + sa.CheckConstraint( + "status != 'prepared' or (executor_id is null and lease_expires_at is null " + "and execution_generation = 0 and terminal_result_code is null " + "and terminal_at is null and replica_id is null and receipt_id is null)", + name="prepared_execution_inactive", + ), + sa.CheckConstraint( + "(producer_request_type = 'guide' and guide_source_item_id is not null " + "and upload_item_id is null and checker_run_id is null and task_id is null " + "and logical_role is null) or " + "(producer_request_type = 'contributor' and guide_source_item_id is null " + "and upload_item_id is not null and checker_run_id is null " + "and task_id is not null and logical_role is null) or " + "(producer_request_type = 'checker_output' and guide_source_item_id is null " + "and upload_item_id is null and checker_run_id is not null " + "and task_id is not null and octet_length(logical_role) between 1 and 100)", + name="producer_reference", + ), + sa.ForeignKeyConstraint( + ["storage_namespace_id", "namespace_fingerprint"], + [ + "artifact_storage_namespaces.id", + "artifact_storage_namespaces.namespace_fingerprint", + ], + name="fk_artifact_put_attempts_namespace_fingerprint", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint(["project_id"], ["projects.id"], ondelete="RESTRICT"), + sa.ForeignKeyConstraint( + ["task_id"], ["workstream_tasks.id"], ondelete="RESTRICT" + ), + sa.ForeignKeyConstraint( + ["guide_source_item_id"], + ["guide_source_snapshot_items.id"], + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["upload_item_id"], ["artifact_upload_items.id"], ondelete="RESTRICT" + ), + sa.ForeignKeyConstraint( + ["checker_run_id"], ["checker_runs.id"], ondelete="RESTRICT" + ), + sa.ForeignKeyConstraint( + ["replica_id"], ["artifact_replicas.id"], ondelete="RESTRICT" + ), + sa.ForeignKeyConstraint( + ["receipt_id"], ["artifact_operation_receipts.id"], ondelete="RESTRICT" + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "operation_identity", + name="uq_artifact_put_attempt_operation", + ), + ) + for column in ( + "project_id", + "task_id", + "guide_source_item_id", + "upload_item_id", + "checker_run_id", + "status", + "next_run_at", + "replica_id", + "receipt_id", + ): + op.create_index( + f"ix_artifact_put_attempts_{column}", + "artifact_put_attempts", + [column], + ) + + op.create_table( + "artifact_put_attempt_charges", + sa.Column("attempt_id", sa.String(36), nullable=False), + sa.Column("charge_id", sa.String(36), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.ForeignKeyConstraint( + ["attempt_id"], ["artifact_put_attempts.id"], ondelete="RESTRICT" + ), + sa.ForeignKeyConstraint( + ["charge_id"], ["artifact_admission_charges.id"], ondelete="RESTRICT" + ), + sa.PrimaryKeyConstraint("attempt_id", "charge_id"), + ) + + +def downgrade() -> None: + """Remove only an empty admission foundation.""" + _refuse_populated_admission_downgrade() + op.drop_table("artifact_put_attempt_charges") + for column in reversed( + ( + "project_id", + "task_id", + "guide_source_item_id", + "upload_item_id", + "checker_run_id", + "status", + "next_run_at", + "replica_id", + "receipt_id", + ) + ): + op.drop_index( + f"ix_artifact_put_attempts_{column}", + table_name="artifact_put_attempts", + ) + op.drop_table("artifact_put_attempts") + op.drop_table("artifact_admission_charges") + op.drop_table("artifact_admission_scopes") + op.drop_constraint( + "uq_artifact_storage_namespace_id_fingerprint", + "artifact_storage_namespaces", + type_="unique", + ) diff --git a/backend/app/core/config.py b/backend/app/core/config.py index 3fa52fc46..b99828474 100644 --- a/backend/app/core/config.py +++ b/backend/app/core/config.py @@ -149,6 +149,10 @@ class Settings(BaseSettings): ) artifact_s3_max_pool_connections: int = Field(default=16, ge=1, le=256) artifact_maximum_bytes: int = Field(default=512 * 1024 * 1024, gt=0) + artifact_admission_task_maximum_bytes: int | None = Field(default=None, gt=0) + artifact_admission_producer_maximum_bytes: int | None = Field(default=None, gt=0) + artifact_admission_project_maximum_bytes: int | None = Field(default=None, gt=0) + artifact_admission_deployment_maximum_bytes: int | None = Field(default=None, gt=0) artifact_stream_buffer_bytes: int = Field(default=1024 * 1024, gt=0, le=1024 * 1024) artifact_operation_lock_timeout_seconds: float = Field( default=1800.0, @@ -433,6 +437,16 @@ def validate_artifact_storage(self) -> Settings: ) if self.artifact_store_backend == "disabled": return self + if any( + value is None + for value in ( + self.artifact_admission_task_maximum_bytes, + self.artifact_admission_producer_maximum_bytes, + self.artifact_admission_project_maximum_bytes, + self.artifact_admission_deployment_maximum_bytes, + ) + ): + raise ValueError("enabled artifact storage requires all durable-byte admission limits") if self.artifact_scratch_root is None: raise ValueError("enabled artifact storage requires an artifact scratch root") if self.artifact_store_backend == "local": diff --git a/backend/app/modules/artifacts/models.py b/backend/app/modules/artifacts/models.py index 2f03ca045..6225b279b 100644 --- a/backend/app/modules/artifacts/models.py +++ b/backend/app/modules/artifacts/models.py @@ -5,9 +5,11 @@ from datetime import datetime from sqlalchemy import ( + BigInteger, CheckConstraint, DateTime, ForeignKey, + ForeignKeyConstraint, Index, Integer, JSON, @@ -21,6 +23,10 @@ SHA256_CHECK = "{column} ~ '^sha256:[0-9a-f]{{64}}$'" +UUID_CHECK = ( + "{column} ~ '^[0-9a-f]{{8}}-[0-9a-f]{{4}}-[1-5][0-9a-f]{{3}}-" + "[89ab][0-9a-f]{{3}}-[0-9a-f]{{12}}$'" +) class ArtifactUploadSession(Base): @@ -213,6 +219,11 @@ class ArtifactStorageNamespace(Base): "namespace_fingerprint", name="uq_artifact_storage_namespace_fingerprint", ), + UniqueConstraint( + "id", + "namespace_fingerprint", + name="uq_artifact_storage_namespace_id_fingerprint", + ), ) id: Mapped[str] = mapped_column(String(20), primary_key=True) @@ -224,6 +235,243 @@ class ArtifactStorageNamespace(Base): created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) +class ArtifactAdmissionScope(Base): + """Serialized durable-byte usage for one canonical admission scope.""" + + __tablename__ = "artifact_admission_scopes" + __table_args__ = ( + CheckConstraint( + "scope_type in ('deployment', 'project', 'producer', 'task')", + name="scope_type", + ), + CheckConstraint("octet_length(scope_id) between 1 and 120", name="scope_id_bounds"), + CheckConstraint("limit_bytes > 0", name="limit_positive"), + CheckConstraint( + "counted_bytes >= 0 and counted_bytes <= limit_bytes", + name="counted_bytes_within_limit", + ), + CheckConstraint("cas_version >= 0", name="cas_nonnegative"), + ) + + scope_type: Mapped[str] = mapped_column(String(20), primary_key=True) + scope_id: Mapped[str] = mapped_column(String(120), primary_key=True) + limit_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False) + counted_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + cas_version: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), server_default=func.now(), onupdate=func.now() + ) + + +class ArtifactAdmissionCharge(Base): + """CAS-protected unique-byte charge for one scope and content identity.""" + + __tablename__ = "artifact_admission_charges" + __table_args__ = ( + ForeignKeyConstraint( + ["scope_type", "scope_id"], + ["artifact_admission_scopes.scope_type", "artifact_admission_scopes.scope_id"], + ondelete="RESTRICT", + name="fk_artifact_admission_charges_scope", + ), + UniqueConstraint( + "scope_type", + "scope_id", + "sha256", + "byte_count", + name="uq_artifact_admission_charge_scope_content", + ), + CheckConstraint(SHA256_CHECK.format(column="sha256"), name="sha256_shape"), + CheckConstraint("byte_count >= 0", name="byte_count_nonnegative"), + CheckConstraint( + "producer_type in ('actor_profile', 'service_identity')", + name="producer_type", + ), + CheckConstraint( + SHA256_CHECK.format(column="creating_operation_identity"), + name="operation_identity_shape", + ), + CheckConstraint( + "state in ('provisional', 'completed', 'released')", + name="state", + ), + CheckConstraint("cas_version >= 0", name="cas_nonnegative"), + CheckConstraint( + "(state = 'completed') = (completed_at is not null)", + name="completed_timestamp", + ), + CheckConstraint( + "state != 'released' or released_at is not null", + name="released_timestamp", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True) + scope_type: Mapped[str] = mapped_column(String(20), nullable=False) + scope_id: Mapped[str] = mapped_column(String(120), nullable=False) + sha256: Mapped[str] = mapped_column(String(71), nullable=False) + byte_count: Mapped[int] = mapped_column(BigInteger, nullable=False) + producer_type: Mapped[str] = mapped_column(String(30), nullable=False) + producer_ref: Mapped[str] = mapped_column(String(120), nullable=False) + creating_operation_identity: Mapped[str] = mapped_column(String(71), nullable=False) + state: Mapped[str] = mapped_column(String(20), nullable=False, default="provisional") + cas_version: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + reserved_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + completed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + released_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), server_default=func.now(), onupdate=func.now() + ) + + +class ArtifactPutAttempt(Base): + """Durable pre-I/O commitment created only after complete admission.""" + + __tablename__ = "artifact_put_attempts" + __table_args__ = ( + ForeignKeyConstraint( + ["storage_namespace_id", "namespace_fingerprint"], + ["artifact_storage_namespaces.id", "artifact_storage_namespaces.namespace_fingerprint"], + ondelete="RESTRICT", + name="fk_artifact_put_attempts_namespace_fingerprint", + ), + UniqueConstraint("operation_identity", name="uq_artifact_put_attempt_operation"), + CheckConstraint( + "producer_request_type in ('guide', 'contributor', 'checker_output')", + name="producer_request_type", + ), + CheckConstraint( + "producer_type in ('actor_profile', 'service_identity')", + name="producer_type", + ), + CheckConstraint( + "((producer_request_type in ('guide', 'contributor') " + "and producer_type = 'actor_profile' and " + + UUID_CHECK.format(column="producer_ref") + + ") or (producer_request_type = 'checker_output' " + "and producer_type = 'service_identity' " + "and producer_ref = 'workstream.artifact.checker_output'))", + name="producer_identity", + ), + CheckConstraint(SHA256_CHECK.format(column="sha256"), name="sha256_shape"), + CheckConstraint("byte_count >= 0", name="byte_count_nonnegative"), + CheckConstraint( + "canonical_target ~ '^sha256/[0-9a-f]{2}/[0-9a-f]{62}$'", + name="canonical_target_shape", + ), + CheckConstraint( + SHA256_CHECK.format(column="operation_identity"), + name="operation_identity_shape", + ), + CheckConstraint( + SHA256_CHECK.format(column="request_digest"), + name="request_digest_shape", + ), + CheckConstraint( + "status in ('prepared', 'put_in_flight', 'acknowledgement_unknown', " + "'object_confirmed', 'absent_replay_required', 'integrity_mismatch', " + "'provider_unavailable', 'conflict')", + name="status", + ), + CheckConstraint( + "(executor_id is null) = (lease_expires_at is null)", + name="executor_lease_pair", + ), + CheckConstraint( + "execution_generation >= 0 and cas_version >= 0", + name="versions_nonnegative", + ), + CheckConstraint( + "status != 'prepared' or (executor_id is null and lease_expires_at is null " + "and execution_generation = 0 and terminal_result_code is null " + "and terminal_at is null and replica_id is null and receipt_id is null)", + name="prepared_execution_inactive", + ), + CheckConstraint( + "(producer_request_type = 'guide' and guide_source_item_id is not null " + "and upload_item_id is null and checker_run_id is null and task_id is null " + "and logical_role is null) or " + "(producer_request_type = 'contributor' and guide_source_item_id is null " + "and upload_item_id is not null and checker_run_id is null and task_id is not null " + "and logical_role is null) or " + "(producer_request_type = 'checker_output' and guide_source_item_id is null " + "and upload_item_id is null and checker_run_id is not null and task_id is not null " + "and octet_length(logical_role) between 1 and 100)", + name="producer_reference", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True) + producer_request_type: Mapped[str] = mapped_column(String(30), nullable=False) + producer_type: Mapped[str] = mapped_column(String(30), nullable=False) + producer_ref: Mapped[str] = mapped_column(String(120), nullable=False) + project_id: Mapped[str] = mapped_column( + ForeignKey("projects.id", ondelete="RESTRICT"), nullable=False, index=True + ) + task_id: Mapped[str | None] = mapped_column( + ForeignKey("workstream_tasks.id", ondelete="RESTRICT"), index=True + ) + guide_source_item_id: Mapped[str | None] = mapped_column( + ForeignKey("guide_source_snapshot_items.id", ondelete="RESTRICT"), index=True + ) + upload_item_id: Mapped[str | None] = mapped_column( + ForeignKey("artifact_upload_items.id", ondelete="RESTRICT"), index=True + ) + checker_run_id: Mapped[str | None] = mapped_column( + ForeignKey("checker_runs.id", ondelete="RESTRICT"), index=True + ) + logical_role: Mapped[str | None] = mapped_column(String(100)) + sha256: Mapped[str] = mapped_column(String(71), nullable=False) + byte_count: Mapped[int] = mapped_column(BigInteger, nullable=False) + media_type: Mapped[str] = mapped_column(String(255), nullable=False) + storage_namespace_id: Mapped[str] = mapped_column(String(20), nullable=False) + namespace_fingerprint: Mapped[str] = mapped_column(String(71), nullable=False) + canonical_target: Mapped[str] = mapped_column(String(1024), nullable=False) + operation_identity: Mapped[str] = mapped_column(String(71), nullable=False) + request_digest: Mapped[str] = mapped_column(String(71), nullable=False) + status: Mapped[str] = mapped_column(String(40), nullable=False, default="prepared", index=True) + next_run_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now(), index=True + ) + executor_id: Mapped[str | None] = mapped_column(String(36)) + lease_expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + execution_generation: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + terminal_result_code: Mapped[str | None] = mapped_column(String(100)) + replica_id: Mapped[str | None] = mapped_column( + ForeignKey("artifact_replicas.id", ondelete="RESTRICT"), index=True + ) + receipt_id: Mapped[str | None] = mapped_column( + ForeignKey("artifact_operation_receipts.id", ondelete="RESTRICT"), index=True + ) + cas_version: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + prepared_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + terminal_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), server_default=func.now(), onupdate=func.now() + ) + + +class ArtifactPutAttemptCharge(Base): + """Immutable link from one put attempt to every required scope charge.""" + + __tablename__ = "artifact_put_attempt_charges" + + attempt_id: Mapped[str] = mapped_column( + ForeignKey("artifact_put_attempts.id", ondelete="RESTRICT"), primary_key=True + ) + charge_id: Mapped[str] = mapped_column( + ForeignKey("artifact_admission_charges.id", ondelete="RESTRICT"), primary_key=True + ) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) + + class ArtifactReplica(Base): """Provider observation record for one immutable content object.""" diff --git a/backend/app/modules/artifacts/repository.py b/backend/app/modules/artifacts/repository.py index a27103365..0b881d662 100644 --- a/backend/app/modules/artifacts/repository.py +++ b/backend/app/modules/artifacts/repository.py @@ -2,18 +2,76 @@ from __future__ import annotations -from sqlalchemy import select +from collections.abc import Sequence +from dataclasses import dataclass +from datetime import datetime + +from sqlalchemy import func, select, tuple_ from sqlalchemy.dialects.postgresql import insert from sqlalchemy.ext.asyncio import AsyncSession +from app.modules.actors.models import ActorIdentityLink, ActorProfile from app.modules.artifacts.models import ( + ArtifactAdmissionCharge, + ArtifactAdmissionScope, ArtifactContent, ArtifactOperationReceipt, + ArtifactPutAttempt, + ArtifactPutAttemptCharge, ArtifactReplica, ArtifactStorageNamespace, ArtifactUploadItem, ArtifactUploadSession, ) +from app.modules.checkers.models import CheckerRun +from app.modules.projects.models import GuideSourceSnapshot, GuideSourceSnapshotItem +from app.modules.tasks.models import WorkstreamTask + + +@dataclass(frozen=True, slots=True) +class GuideAdmissionFacts: + """Authoritative project ownership for one guide source item.""" + + guide_source_item_id: str + project_id: str + content_hash: str + media_type: str + + +@dataclass(frozen=True, slots=True) +class ContributorAdmissionFacts: + """Authoritative upload-item ownership and state.""" + + upload_item_id: str + project_id: str + task_id: str | None + actor_profile_id: str + session_state: str + item_state: str + expected_sha256: str | None + expected_size: int | None + media_type: str | None + + +@dataclass(frozen=True, slots=True) +class CheckerOutputAdmissionFacts: + """Authoritative project/task ownership for one checker run.""" + + checker_run_id: str + project_id: str + task_id: str + + +@dataclass(frozen=True, slots=True) +class ServiceActorFacts: + """Locked service profile and identity-link state.""" + + actor_profile_id: str + actor_kind: str + actor_status: str + service_identity: str | None + identity_link_id: str + identity_link_status: str class ArtifactRepository: @@ -23,6 +81,13 @@ def __init__(self, session: AsyncSession) -> None: """Bind the repository to one async database session.""" self._session = session + async def database_now(self) -> datetime: + """Return the PostgreSQL clock for admission timestamps.""" + value = await self._session.scalar(select(func.clock_timestamp())) + if value is None: + raise RuntimeError("PostgreSQL clock did not return a timestamp") + return value + async def lock_upload_item(self, item_id: str) -> ArtifactUploadItem | None: """Load one upload item with a row lock.""" result = await self._session.execute( @@ -41,6 +106,219 @@ async def lock_upload_session(self, session_id: str) -> ArtifactUploadSession | ) return result.scalar_one_or_none() + async def get_guide_admission_facts( + self, guide_source_item_id: str + ) -> GuideAdmissionFacts | None: + """Load canonical project ownership for one guide source item.""" + row = ( + await self._session.execute( + select( + GuideSourceSnapshotItem.id, + GuideSourceSnapshot.project_id, + GuideSourceSnapshotItem.content_hash, + GuideSourceSnapshotItem.media_type, + ) + .join( + GuideSourceSnapshot, + GuideSourceSnapshot.id == GuideSourceSnapshotItem.source_snapshot_id, + ) + .where(GuideSourceSnapshotItem.id == guide_source_item_id) + ) + ).one_or_none() + if row is None: + return None + return GuideAdmissionFacts( + guide_source_item_id=row.id, + project_id=row.project_id, + content_hash=row.content_hash, + media_type=row.media_type, + ) + + async def get_contributor_admission_facts( + self, upload_item_id: str + ) -> ContributorAdmissionFacts | None: + """Load canonical contributor upload ownership and state.""" + row = ( + await self._session.execute( + select( + ArtifactUploadItem.id, + ArtifactUploadSession.project_id, + ArtifactUploadSession.task_id, + ArtifactUploadSession.actor_id, + ArtifactUploadSession.state.label("session_state"), + ArtifactUploadItem.state.label("item_state"), + ArtifactUploadItem.expected_sha256, + ArtifactUploadItem.expected_size, + ArtifactUploadItem.media_type, + ) + .join( + ArtifactUploadSession, + ArtifactUploadSession.id == ArtifactUploadItem.session_id, + ) + .where(ArtifactUploadItem.id == upload_item_id) + .with_for_update(of=(ArtifactUploadSession, ArtifactUploadItem)) + ) + ).one_or_none() + if row is None: + return None + return ContributorAdmissionFacts( + upload_item_id=row.id, + project_id=row.project_id, + task_id=row.task_id, + actor_profile_id=row.actor_id, + session_state=row.session_state, + item_state=row.item_state, + expected_sha256=row.expected_sha256, + expected_size=row.expected_size, + media_type=row.media_type, + ) + + async def get_checker_output_admission_facts( + self, checker_run_id: str + ) -> CheckerOutputAdmissionFacts | None: + """Load canonical project/task ownership for one checker run.""" + row = ( + await self._session.execute( + select(CheckerRun.id, CheckerRun.task_id, WorkstreamTask.project_id) + .join(WorkstreamTask, WorkstreamTask.id == CheckerRun.task_id) + .where(CheckerRun.id == checker_run_id) + ) + ).one_or_none() + if row is None: + return None + return CheckerOutputAdmissionFacts( + checker_run_id=row.id, + project_id=row.project_id, + task_id=row.task_id, + ) + + async def lock_service_actor(self, actor_profile_id: str) -> ServiceActorFacts | None: + """Lock one canonical service profile and its exact identity link.""" + row = ( + await self._session.execute( + select( + ActorProfile.id, + ActorProfile.actor_kind, + ActorProfile.status.label("actor_status"), + ActorProfile.service_identity, + ActorIdentityLink.id.label("identity_link_id"), + ActorIdentityLink.status.label("identity_link_status"), + ) + .join(ActorIdentityLink, ActorIdentityLink.actor_profile_id == ActorProfile.id) + .where(ActorProfile.id == actor_profile_id) + .with_for_update(of=(ActorProfile, ActorIdentityLink)) + ) + ).one_or_none() + if row is None: + return None + return ServiceActorFacts( + actor_profile_id=row.id, + actor_kind=row.actor_kind, + actor_status=row.actor_status, + service_identity=row.service_identity, + identity_link_id=row.identity_link_id, + identity_link_status=row.identity_link_status, + ) + + async def ensure_and_lock_admission_scopes( + self, + scopes: Sequence[tuple[str, str, int]], + ) -> tuple[ArtifactAdmissionScope, ...]: + """Create missing counters, then lock every scope in canonical order.""" + values = [ + { + "scope_type": scope_type, + "scope_id": scope_id, + "limit_bytes": limit_bytes, + "counted_bytes": 0, + "cas_version": 0, + } + for scope_type, scope_id, limit_bytes in scopes + ] + await self._session.execute( + insert(ArtifactAdmissionScope) + .values(values) + .on_conflict_do_nothing( + index_elements=[ + ArtifactAdmissionScope.scope_type, + ArtifactAdmissionScope.scope_id, + ] + ) + ) + keys = [(scope_type, scope_id) for scope_type, scope_id, _ in scopes] + result = await self._session.execute( + select(ArtifactAdmissionScope) + .where( + tuple_( + ArtifactAdmissionScope.scope_type, + ArtifactAdmissionScope.scope_id, + ).in_(keys) + ) + .order_by(ArtifactAdmissionScope.scope_type, ArtifactAdmissionScope.scope_id) + .with_for_update() + ) + return tuple(result.scalars().all()) + + async def get_admission_charge( + self, + *, + scope_type: str, + scope_id: str, + sha256: str, + byte_count: int, + ) -> ArtifactAdmissionCharge | None: + """Load one exact scope/content charge while its scope is locked.""" + return await self._session.scalar( + select(ArtifactAdmissionCharge).where( + ArtifactAdmissionCharge.scope_type == scope_type, + ArtifactAdmissionCharge.scope_id == scope_id, + ArtifactAdmissionCharge.sha256 == sha256, + ArtifactAdmissionCharge.byte_count == byte_count, + ) + ) + + async def add_admission_charge( + self, charge: ArtifactAdmissionCharge + ) -> ArtifactAdmissionCharge: + """Flush one new charge under its locked scope counter.""" + self._session.add(charge) + await self._session.flush() + return charge + + async def get_put_attempt_by_operation( + self, operation_identity: str + ) -> ArtifactPutAttempt | None: + """Load the durable attempt for one canonical operation identity.""" + return await self._session.scalar( + select(ArtifactPutAttempt).where( + ArtifactPutAttempt.operation_identity == operation_identity + ) + ) + + async def add_put_attempt( + self, + attempt: ArtifactPutAttempt, + charges: Sequence[ArtifactAdmissionCharge], + ) -> ArtifactPutAttempt: + """Flush one attempt and its complete charge links in this transaction.""" + self._session.add(attempt) + await self._session.flush() + self._session.add_all( + ArtifactPutAttemptCharge(attempt_id=attempt.id, charge_id=charge.id) + for charge in charges + ) + await self._session.flush() + return attempt + + async def list_put_attempt_charge_ids(self, attempt_id: str) -> tuple[str, ...]: + """Return one attempt's charge IDs in stable order.""" + result = await self._session.execute( + select(ArtifactPutAttemptCharge.charge_id) + .where(ArtifactPutAttemptCharge.attempt_id == attempt_id) + .order_by(ArtifactPutAttemptCharge.charge_id) + ) + return tuple(result.scalars().all()) + async def get_or_create_content(self, content: ArtifactContent) -> ArtifactContent: """Return the immutable content fact for one digest and size.""" await self._session.execute( diff --git a/backend/app/modules/artifacts/schemas.py b/backend/app/modules/artifacts/schemas.py new file mode 100644 index 000000000..5370e1cef --- /dev/null +++ b/backend/app/modules/artifacts/schemas.py @@ -0,0 +1,61 @@ +"""Closed internal contracts for durable artifact admission.""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TypeAlias, final +from uuid import UUID + +from app.modules.artifacts.sources import CommittedArtifactSource +from app.modules.authorization.runtime import AuthorizationContext + + +@final +@dataclass(frozen=True, slots=True) +class GuideArtifactAdmissionRequest: + """One prepared guide source item admitted under its canonical project.""" + + authorization_context: AuthorizationContext + guide_source_item_id: UUID + source: CommittedArtifactSource + + +@final +@dataclass(frozen=True, slots=True) +class ContributorArtifactAdmissionRequest: + """One prepared contributor item admitted under its upload session.""" + + authorization_context: AuthorizationContext + upload_item_id: UUID + source: CommittedArtifactSource + + +@final +@dataclass(frozen=True, slots=True) +class CheckerOutputArtifactAdmissionRequest: + """One prepared checker output admitted under its exact checker run.""" + + authorization_context: AuthorizationContext + checker_run_id: UUID + logical_role: str + source: CommittedArtifactSource + + +ArtifactAdmissionRequest: TypeAlias = ( + GuideArtifactAdmissionRequest + | ContributorArtifactAdmissionRequest + | CheckerOutputArtifactAdmissionRequest +) + + +@final +@dataclass(frozen=True, slots=True) +class ArtifactAdmissionResult: + """Committed pre-I/O attempt and its complete admission-charge set.""" + + attempt_id: UUID + status: str + operation_identity: str + request_digest: str + charge_ids: tuple[UUID, ...] + replayed: bool diff --git a/backend/app/modules/artifacts/service.py b/backend/app/modules/artifacts/service.py index 72e1903da..9938c2c64 100644 --- a/backend/app/modules/artifacts/service.py +++ b/backend/app/modules/artifacts/service.py @@ -2,36 +2,44 @@ from __future__ import annotations -import asyncio from dataclasses import dataclass -from uuid import uuid4 +from uuid import UUID, uuid4 from sqlalchemy.ext.asyncio import AsyncSession +from app.adapters.artifacts.references import artifact_provider_object_ref from app.core.config import Settings -from app.core.cancellation import await_cancellation_resistant +from app.core.hashing import canonical_json_hash from app.db.session import get_session_factory from app.interfaces.artifacts import ( - ArtifactInputMismatchError, - ArtifactIntegrityError, - ArtifactPutResult, ArtifactStore, ArtifactStoreBootstrap, - ArtifactStoreError, ArtifactStoreNamespaceClaim, artifact_store_namespace_material, ) from app.interfaces.external_services import ExternalServiceAdapterIdentity +from app.modules.actors.service_identities import ServiceIdentity from app.modules.artifacts.models import ( - ArtifactContent, - ArtifactOperationReceipt, - ArtifactReplica, + ArtifactAdmissionCharge, + ArtifactAdmissionScope, + ArtifactPutAttempt, ArtifactStorageNamespace, - ArtifactUploadItem, - ArtifactUploadSession, ) from app.modules.artifacts.repository import ArtifactRepository -from app.modules.artifacts.sources import ArtifactCommitment, CommittedArtifactSource +from app.modules.artifacts.schemas import ( + ArtifactAdmissionRequest, + ArtifactAdmissionResult, + CheckerOutputArtifactAdmissionRequest, + ContributorArtifactAdmissionRequest, + GuideArtifactAdmissionRequest, +) +from app.modules.artifacts.sources import CommittedArtifactSource +from app.modules.authorization.runtime import ( + ActorKind, + ActorStatus, + AuthorizationContext, + IdentityLinkStatus, +) ARTIFACT_STORAGE_NAMESPACE_ID = "primary" @@ -56,11 +64,49 @@ class ArtifactStorageNamespaceSpec: namespace_fingerprint: str +class ArtifactAdmissionError(ArtifactIngestStateError): + """Base failure for durable-byte admission before provider I/O.""" + + +class ArtifactAdmissionConfigurationError(ArtifactAdmissionError): + """Raised when admission configuration is absent or has drifted.""" + + +class ArtifactAdmissionCapacityError(ArtifactAdmissionError): + """Raised when one required durable-byte scope lacks capacity.""" + + +class ArtifactAdmissionConflictError(ArtifactAdmissionError): + """Raised when an operation identity is replayed with changed input.""" + + +class ArtifactAdmissionRelationshipError(ArtifactAdmissionError): + """Raised when canonical producer ownership cannot be derived.""" + + @dataclass(frozen=True, slots=True) -class ProviderAttemptFence: - """Committed item CAS value fencing one provider call.""" +class _AdmissionScopeSpec: + """One server-derived scope and its exact configured byte limit.""" - item_cas: int + scope_type: str + scope_id: str + limit_bytes: int + + +@dataclass(frozen=True, slots=True) +class _AdmissionFacts: + """Canonical producer and product facts loaded for one closed request.""" + + request_type: str + producer_type: str + producer_ref: str + project_id: str + task_id: str | None + guide_source_item_id: str | None + upload_item_id: str | None + checker_run_id: str | None + logical_role: str | None + operation_identity: str def artifact_storage_namespace_spec( @@ -89,7 +135,7 @@ def artifact_storage_namespace_spec( class ArtifactStorageOrchestrator: - """Sole internal owner of the writable ArtifactStore capability.""" + """Dormant owner of writable storage until 02C2 adds attempt execution.""" def __init__( self, @@ -108,250 +154,441 @@ async def ensure_storage_namespace(self) -> ArtifactStorageNamespace: async with self._session.begin(): return await self._claim_and_validate_namespace() - async def put_reserved_item( - self, - item_id: str, - source: CommittedArtifactSource, - *, - correlation_id: str | None = None, - ) -> ArtifactPutResult: - """Publish one already-reserved internal item without activating product ingest.""" - commitment = source.commitment - fence = await self._start_put(item_id, commitment) - try: - result = await self._store.put(source) - except asyncio.CancelledError as cancellation: - try: - await await_cancellation_resistant( - self._mark_replay_required(item_id, fence) - ) - except BaseException: - raise cancellation from None - raise - except ArtifactStoreError as exc: - if exc.retryable: - await self._mark_replay_required(item_id, fence) - else: - await self._fail_put(item_id, exc.code, fence) - raise - except BaseException: - await self._mark_replay_required(item_id, fence) - raise - - try: - await self._finalize_put( - item_id, - commitment, - result, - fence, - correlation_id=correlation_id or str(uuid4()), - ) - except (ArtifactIntegrityError, ArtifactInputMismatchError) as exc: - await self._fail_put(item_id, exc.code, fence) - raise - except asyncio.CancelledError as cancellation: - try: - await await_cancellation_resistant( - self._mark_replay_required(item_id, fence) - ) - except BaseException: - raise cancellation from None - raise - except BaseException: - await self._mark_replay_required(item_id, fence) - raise - return result - - async def _start_put( - self, - item_id: str, - commitment: ArtifactCommitment, - ) -> ProviderAttemptFence: - """Transaction A validates namespace and exact reserved commitment.""" - async with self._session.begin(): - await self._claim_and_validate_namespace() - item = await self._repo.lock_upload_item(item_id) - if item is None: - raise ArtifactIngestStateError("artifact upload item does not exist") - upload_session = await self._repo.lock_upload_session(item.session_id) - if upload_session is None or upload_session.state != "open": - raise ArtifactIngestStateError("artifact upload session is not open") - if item.state not in {"reserved", "replay_required"}: - raise ArtifactIngestStateError("artifact upload item cannot be stored") - if ( - item.expected_sha256 != commitment.sha256 - or item.expected_size != commitment.byte_count - or item.media_type != commitment.media_type - or commitment.byte_count > item.reserved_bytes - ): - raise ArtifactIngestStateError("artifact upload commitment changed") - item.state = "uploading" - item.error_code = None - item.cas_version += 1 - upload_session.cas_version += 1 - return ProviderAttemptFence(item.cas_version) - - async def _finalize_put( + async def _claim_and_validate_namespace(self) -> ArtifactStorageNamespace: + """Atomically claim the singleton or reject deployment identity drift.""" + return await _claim_and_validate_storage_namespace(self._repo, self._namespace) + + +class ArtifactAdmissionService: + """Create one fully admitted put attempt without provider execution.""" + + def __init__( self, - item_id: str, - commitment: ArtifactCommitment, - result: ArtifactPutResult, - fence: ProviderAttemptFence, - *, - correlation_id: str, + session: AsyncSession, + settings: Settings, + namespace: ArtifactStorageNamespaceSpec, ) -> None: - """Transaction B records acknowledgement without making bytes bindable.""" + """Bind admission to one transaction owner and configured namespace.""" + self._session = session + self._settings = settings + self._namespace = namespace + self._repo = ArtifactRepository(session) + + async def admit( + self, + request: ArtifactAdmissionRequest, + ) -> ArtifactAdmissionResult: + """Reserve every derived scope and persist one prepared attempt atomically.""" + self._validate_request_boundary(request) + commitment = request.source.commitment async with self._session.begin(): - namespace = await self._claim_and_validate_namespace() - item, upload_session = await self._locked_attempt(item_id, fence) - existing_receipt = await self._repo.get_receipt_for_item(item.id) - if existing_receipt is not None: - raise ArtifactIntegrityError("artifact upload item already has put evidence") - - content = await self._repo.get_or_create_content( - ArtifactContent( - id=str(uuid4()), - sha256=commitment.sha256, - byte_count=commitment.byte_count, - media_type=commitment.media_type, - normalized_display_name=item.display_name, - ) - ) - replica = await self._repo.get_or_create_replica( - ArtifactReplica( - id=str(uuid4()), - content_id=content.id, - storage_namespace_id=namespace.id, - namespace_fingerprint=namespace.namespace_fingerprint, - adapter=namespace.adapter, - provider_profile=namespace.provider_profile, - provider_object_ref=result.provider_object_ref, - verification_state="pending", - availability_state="unknown", - integrity_state="unknown", - ) + namespace = await _claim_and_validate_storage_namespace( + self._repo, + self._namespace, ) - if ( - replica.content_id != content.id - or replica.namespace_fingerprint != namespace.namespace_fingerprint - or replica.adapter != namespace.adapter - or replica.provider_profile != namespace.provider_profile - ): - raise ArtifactIntegrityError("provider object reference changed content identity") - - await self._repo.add_receipt( - ArtifactOperationReceipt( - id=str(uuid4()), - upload_item_id=item.id, - replica_id=replica.id, - operation="put", - idempotency_key=item.idempotency_key, - request_digest=item.request_digest, - provider_object_ref=result.provider_object_ref, - replayed=result.replayed, - outcome="stored_pending_verification", - attempt_number=1, - correlation_id=correlation_id, - details=[ + facts = await self._derive_admission_facts(request) + scopes = self._derive_scopes(facts) + request_digest = canonical_json_hash( + { + "operation_identity": facts.operation_identity, + "request_type": facts.request_type, + "producer_type": facts.producer_type, + "producer_ref": facts.producer_ref, + "project_id": facts.project_id, + "task_id": facts.task_id, + "guide_source_item_id": facts.guide_source_item_id, + "upload_item_id": facts.upload_item_id, + "checker_run_id": facts.checker_run_id, + "logical_role": facts.logical_role, + "sha256": commitment.sha256, + "byte_count": commitment.byte_count, + "media_type": commitment.media_type, + "namespace_fingerprint": namespace.namespace_fingerprint, + "scopes": [ { - "name": "namespace_fingerprint", - "value": namespace.namespace_fingerprint, + "scope_type": scope.scope_type, + "scope_id": scope.scope_id, + "limit_bytes": scope.limit_bytes, } + for scope in scopes ], - ) + } ) - item.state = "stored_pending_verification" - item.content_id = content.id - item.provider_object_ref = result.provider_object_ref - item.cas_version += 1 - self._apply_committed_accounting(upload_session, item, commitment.byte_count) - - async def _claim_and_validate_namespace(self) -> ArtifactStorageNamespace: - """Atomically claim the singleton or reject deployment identity drift.""" - return await _claim_and_validate_storage_namespace(self._repo, self._namespace) + replay = await self._existing_attempt_result( + facts.operation_identity, + request_digest, + ) + if replay is not None: + return replay + + counters = await self._repo.ensure_and_lock_admission_scopes( + [ + (scope.scope_type, scope.scope_id, scope.limit_bytes) + for scope in scopes + ] + ) + # A concurrent first caller may have committed while these shared + # scope locks were pending. Recheck under serialization before any + # counter or charge mutation. + replay = await self._existing_attempt_result( + facts.operation_identity, + request_digest, + ) + if replay is not None: + return replay + charges = await self._reserve_charges( + scopes=scopes, + counters=counters, + facts=facts, + sha256=commitment.sha256, + byte_count=commitment.byte_count, + ) + database_now = await self._repo.database_now() + attempt = ArtifactPutAttempt( + id=str(uuid4()), + producer_request_type=facts.request_type, + producer_type=facts.producer_type, + producer_ref=facts.producer_ref, + project_id=facts.project_id, + task_id=facts.task_id, + guide_source_item_id=facts.guide_source_item_id, + upload_item_id=facts.upload_item_id, + checker_run_id=facts.checker_run_id, + logical_role=facts.logical_role, + sha256=commitment.sha256, + byte_count=commitment.byte_count, + media_type=commitment.media_type, + storage_namespace_id=namespace.id, + namespace_fingerprint=namespace.namespace_fingerprint, + canonical_target=artifact_provider_object_ref(commitment), + operation_identity=facts.operation_identity, + request_digest=request_digest, + status="prepared", + next_run_at=database_now, + executor_id=None, + lease_expires_at=None, + execution_generation=0, + terminal_result_code=None, + replica_id=None, + receipt_id=None, + cas_version=0, + prepared_at=database_now, + terminal_at=None, + ) + await self._repo.add_put_attempt(attempt, charges) + return await self._result(attempt, replayed=False) - async def _locked_attempt( - self, - item_id: str, - fence: ProviderAttemptFence, - ) -> tuple[ArtifactUploadItem, ArtifactUploadSession]: - """Reload and fence the exact item/session pair after provider I/O.""" - item = await self._repo.lock_upload_item(item_id) - if item is None or item.state != "uploading" or item.cas_version != fence.item_cas: - raise ArtifactIngestStateError("artifact upload item is not awaiting acknowledgement") - upload_session = await self._repo.lock_upload_session(item.session_id) + @staticmethod + def _validate_request_boundary(request: ArtifactAdmissionRequest) -> None: + """Reject open-ended or forged internal request shapes.""" + if type(request) not in { + GuideArtifactAdmissionRequest, + ContributorArtifactAdmissionRequest, + CheckerOutputArtifactAdmissionRequest, + }: + raise TypeError("invalid artifact admission request") + if type(request.authorization_context) is not AuthorizationContext: + raise TypeError("invalid artifact admission authorization context") + if type(request.source) is not CommittedArtifactSource: + raise TypeError("invalid artifact admission source") + context = request.authorization_context if ( - upload_session is None - or upload_session.state != "open" + context.actor_status is not ActorStatus.ACTIVE + or context.identity_link_status is not IdentityLinkStatus.ACTIVE ): - raise ArtifactIngestStateError("artifact upload session is not open") - return item, upload_session + raise ArtifactAdmissionRelationshipError( + "artifact admission actor is not active" + ) - @staticmethod - def _apply_committed_accounting( - upload_session: ArtifactUploadSession, - item: ArtifactUploadItem, - byte_count: int, - ) -> None: - """Move one item from reserved capacity to acknowledged byte usage.""" - ArtifactStorageOrchestrator._validate_reserved_accounting(upload_session, item) - upload_session.reserved_bytes -= item.reserved_bytes - upload_session.reserved_items -= 1 - upload_session.current_bytes += byte_count - upload_session.current_items += 1 - upload_session.cas_version += 1 - - async def _mark_replay_required( - self, - item_id: str, - fence: ProviderAttemptFence, - ) -> None: - """Record an ambiguous acknowledgement only when the attempt fence matches.""" - await self._session.rollback() - async with self._session.begin(): - item = await self._repo.lock_upload_item(item_id) - if item is None or item.state != "uploading" or item.cas_version != fence.item_cas: - return - item.state = "replay_required" - item.error_code = "artifact_put_acknowledgement_unknown" - item.cas_version += 1 - - async def _fail_put( - self, - item_id: str, - error_code: str, - fence: ProviderAttemptFence, - ) -> None: - """Fail one terminal provider attempt and release its reservation once.""" - await self._session.rollback() - async with self._session.begin(): - item = await self._repo.lock_upload_item(item_id) - if item is None or item.state != "uploading" or item.cas_version != fence.item_cas: - return - upload_session = await self._repo.lock_upload_session(item.session_id) - if upload_session is None: - raise ArtifactIntegrityError("artifact upload session is missing") - self._validate_reserved_accounting(upload_session, item) - item.state = "failed" - item.error_code = error_code - item.cas_version += 1 - upload_session.reserved_bytes -= item.reserved_bytes - upload_session.reserved_items -= 1 - upload_session.cas_version += 1 + async def _derive_admission_facts( + self, request: ArtifactAdmissionRequest + ) -> _AdmissionFacts: + """Load every product and producer relationship from authoritative rows.""" + if type(request) is GuideArtifactAdmissionRequest: + return await self._guide_facts(request) + if type(request) is ContributorArtifactAdmissionRequest: + return await self._contributor_facts(request) + if type(request) is CheckerOutputArtifactAdmissionRequest: + return await self._checker_output_facts(request) + raise TypeError("invalid artifact admission request") + + async def _guide_facts( + self, request: GuideArtifactAdmissionRequest + ) -> _AdmissionFacts: + """Bind committed bytes to one authoritative guide source item.""" + context = request.authorization_context + if context.actor_kind is not ActorKind.HUMAN: + raise ArtifactAdmissionRelationshipError( + "guide artifact producer must be a human actor" + ) + item_id = str(request.guide_source_item_id) + row = await self._repo.get_guide_admission_facts(item_id) + commitment = request.source.commitment + if ( + row is None + or row.content_hash != commitment.sha256 + or row.media_type != commitment.media_type + ): + raise ArtifactAdmissionRelationshipError( + "guide source item relationship is unavailable" + ) + operation_identity = canonical_json_hash( + {"request_type": "guide", "guide_source_item_id": item_id} + ) + return _AdmissionFacts( + request_type="guide", + producer_type="actor_profile", + producer_ref=str(context.actor_profile_id), + project_id=row.project_id, + task_id=None, + guide_source_item_id=item_id, + upload_item_id=None, + checker_run_id=None, + logical_role=None, + operation_identity=operation_identity, + ) + + async def _contributor_facts( + self, request: ContributorArtifactAdmissionRequest + ) -> _AdmissionFacts: + """Bind committed bytes to one contributor-owned upload item.""" + context = request.authorization_context + if context.actor_kind is not ActorKind.HUMAN: + raise ArtifactAdmissionRelationshipError( + "contributor artifact producer must be a human actor" + ) + item_id = str(request.upload_item_id) + row = await self._repo.get_contributor_admission_facts(item_id) + commitment = request.source.commitment + if ( + row is None + or row.actor_profile_id != str(context.actor_profile_id) + or row.task_id is None + or row.session_state != "open" + or row.item_state not in {"reserved", "replay_required"} + or row.expected_sha256 != commitment.sha256 + or row.expected_size != commitment.byte_count + or row.media_type != commitment.media_type + ): + raise ArtifactAdmissionRelationshipError( + "contributor upload item relationship is unavailable" + ) + operation_identity = canonical_json_hash( + {"request_type": "contributor", "upload_item_id": item_id} + ) + return _AdmissionFacts( + request_type="contributor", + producer_type="actor_profile", + producer_ref=str(context.actor_profile_id), + project_id=row.project_id, + task_id=row.task_id, + guide_source_item_id=None, + upload_item_id=item_id, + checker_run_id=None, + logical_role=None, + operation_identity=operation_identity, + ) + + async def _checker_output_facts( + self, request: CheckerOutputArtifactAdmissionRequest + ) -> _AdmissionFacts: + """Bind committed bytes to one run and fixed checker service actor.""" + context = request.authorization_context + if context.actor_kind is not ActorKind.SERVICE: + raise ArtifactAdmissionRelationshipError( + "checker output producer must be a service actor" + ) + logical_role = self._validate_logical_role(request.logical_role) + service_actor = await self._repo.lock_service_actor( + str(context.actor_profile_id) + ) + if ( + service_actor is None + or service_actor.actor_kind != "service" + or service_actor.actor_status != "active" + or service_actor.identity_link_id != str(context.identity_link_id) + or service_actor.identity_link_status != "active" + or service_actor.service_identity + != ServiceIdentity.ARTIFACT_CHECKER_OUTPUT.value + ): + raise ArtifactAdmissionRelationshipError( + "checker output service identity is unavailable" + ) + checker_run_id = str(request.checker_run_id) + row = await self._repo.get_checker_output_admission_facts(checker_run_id) + if row is None: + raise ArtifactAdmissionRelationshipError( + "checker run relationship is unavailable" + ) + operation_identity = canonical_json_hash( + { + "request_type": "checker_output", + "checker_run_id": checker_run_id, + "logical_role": logical_role, + } + ) + return _AdmissionFacts( + request_type="checker_output", + producer_type="service_identity", + producer_ref=ServiceIdentity.ARTIFACT_CHECKER_OUTPUT.value, + project_id=row.project_id, + task_id=row.task_id, + guide_source_item_id=None, + upload_item_id=None, + checker_run_id=checker_run_id, + logical_role=logical_role, + operation_identity=operation_identity, + ) @staticmethod - def _validate_reserved_accounting( - upload_session: ArtifactUploadSession, - item: ArtifactUploadItem, - ) -> None: - """Reject aggregate drift before consuming one item's reservation.""" + def _validate_logical_role(value: str) -> str: + """Require one bounded printable checker-output role.""" if ( - upload_session.reserved_bytes < item.reserved_bytes - or upload_session.reserved_items < 1 + not isinstance(value, str) + or value != value.strip() + or not value + or len(value) > 100 + or any(ord(character) < 32 or ord(character) == 127 for character in value) + ): + raise ArtifactAdmissionRelationshipError( + "checker output logical role is invalid" + ) + return value + + def _derive_scopes(self, facts: _AdmissionFacts) -> tuple[_AdmissionScopeSpec, ...]: + """Derive the complete closed scope set without caller participation.""" + limits = self._configured_limits() + scopes = [ + _AdmissionScopeSpec( + "deployment", + ARTIFACT_STORAGE_NAMESPACE_ID, + limits["deployment"], + ), + _AdmissionScopeSpec("project", facts.project_id, limits["project"]), + _AdmissionScopeSpec( + "producer", + f"{facts.producer_type}:{facts.producer_ref}", + limits["producer"], + ), + ] + if facts.task_id is not None: + scopes.append(_AdmissionScopeSpec("task", facts.task_id, limits["task"])) + return tuple(sorted(scopes, key=lambda value: (value.scope_type, value.scope_id))) + + def _configured_limits(self) -> dict[str, int]: + """Return exact positive limits only for an enabled artifact backend.""" + values = { + "task": self._settings.artifact_admission_task_maximum_bytes, + "producer": self._settings.artifact_admission_producer_maximum_bytes, + "project": self._settings.artifact_admission_project_maximum_bytes, + "deployment": self._settings.artifact_admission_deployment_maximum_bytes, + } + if self._settings.artifact_store_backend == "disabled" or any( + type(value) is not int or value <= 0 for value in values.values() ): - raise ArtifactIntegrityError( - "artifact upload session reservation accounting is invalid" + raise ArtifactAdmissionConfigurationError( + "artifact durable-byte admission is not configured" + ) + return {key: int(value) for key, value in values.items()} + + async def _existing_attempt_result( + self, + operation_identity: str, + request_digest: str, + ) -> ArtifactAdmissionResult | None: + """Return an exact replay or reject changed input for one operation.""" + existing = await self._repo.get_put_attempt_by_operation(operation_identity) + if existing is None: + return None + if existing.request_digest != request_digest: + raise ArtifactAdmissionConflictError( + "artifact admission operation input changed" ) + return await self._result(existing, replayed=True) + + async def _reserve_charges( + self, + *, + scopes: tuple[_AdmissionScopeSpec, ...], + counters: tuple[ArtifactAdmissionScope, ...], + facts: _AdmissionFacts, + sha256: str, + byte_count: int, + ) -> tuple[ArtifactAdmissionCharge, ...]: + """Reserve unique content under every locked scope or fail atomically.""" + counter_by_key = { + (counter.scope_type, counter.scope_id): counter for counter in counters + } + if len(counter_by_key) != len(scopes): + raise ArtifactAdmissionConfigurationError( + "artifact admission scope set is incomplete" + ) + database_now = await self._repo.database_now() + charges: list[ArtifactAdmissionCharge] = [] + for scope in scopes: + counter = counter_by_key[(scope.scope_type, scope.scope_id)] + if counter.limit_bytes != scope.limit_bytes: + raise ArtifactAdmissionConfigurationError( + "artifact admission scope limit does not match configuration" + ) + charge = await self._repo.get_admission_charge( + scope_type=scope.scope_type, + scope_id=scope.scope_id, + sha256=sha256, + byte_count=byte_count, + ) + if charge is not None and charge.state in {"provisional", "completed"}: + charges.append(charge) + continue + if counter.counted_bytes + byte_count > counter.limit_bytes: + raise ArtifactAdmissionCapacityError( + f"artifact durable-byte limit exceeded for {scope.scope_type} scope" + ) + counter.counted_bytes += byte_count + counter.cas_version += 1 + if charge is None: + charge = await self._repo.add_admission_charge( + ArtifactAdmissionCharge( + id=str(uuid4()), + scope_type=scope.scope_type, + scope_id=scope.scope_id, + sha256=sha256, + byte_count=byte_count, + producer_type=facts.producer_type, + producer_ref=facts.producer_ref, + creating_operation_identity=facts.operation_identity, + state="provisional", + cas_version=0, + reserved_at=database_now, + completed_at=None, + released_at=None, + ) + ) + elif charge.state == "released": + charge.state = "provisional" + charge.reserved_at = database_now + charge.released_at = None + charge.cas_version += 1 + else: + raise ArtifactAdmissionConflictError( + "artifact admission charge state is invalid" + ) + charges.append(charge) + return tuple(charges) + + async def _result( + self, attempt: ArtifactPutAttempt, *, replayed: bool + ) -> ArtifactAdmissionResult: + """Return one detached-safe internal result.""" + charge_ids = await self._repo.list_put_attempt_charge_ids(attempt.id) + return ArtifactAdmissionResult( + attempt_id=UUID(attempt.id), + status=attempt.status, + operation_identity=attempt.operation_identity, + request_digest=attempt.request_digest, + charge_ids=tuple(UUID(charge_id) for charge_id in charge_ids), + replayed=replayed, + ) async def validate_artifact_storage_namespace_at_startup( diff --git a/backend/tests/artifact_store_helpers.py b/backend/tests/artifact_store_helpers.py index bdc149a35..10a2f02c2 100644 --- a/backend/tests/artifact_store_helpers.py +++ b/backend/tests/artifact_store_helpers.py @@ -20,6 +20,18 @@ from app.modules.artifacts.sources import CommittedArtifactSource +def artifact_admission_limit_settings( + maximum_bytes: int = 1024, +) -> dict[str, int]: + """Return explicit bounded admission limits for enabled test stores.""" + return { + "artifact_admission_task_maximum_bytes": maximum_bytes, + "artifact_admission_producer_maximum_bytes": maximum_bytes, + "artifact_admission_project_maximum_bytes": maximum_bytes, + "artifact_admission_deployment_maximum_bytes": maximum_bytes, + } + + async def artifact_byte_stream(*chunks: bytes) -> AsyncIterator[bytes]: """Yield exact test bytes through the public preparation boundary.""" for chunk in chunks: diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py new file mode 100644 index 000000000..149b13086 --- /dev/null +++ b/backend/tests/test_artifact_admission.py @@ -0,0 +1,895 @@ +"""PostgreSQL proofs for atomic durable-byte admission before provider I/O.""" + +from __future__ import annotations + +import asyncio +from datetime import UTC, datetime, timedelta +from pathlib import Path +from unittest.mock import AsyncMock +from uuid import UUID, uuid4 + +from alembic import command +from alembic.config import Config +import pytest +from sqlalchemy import func, select, text +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from app.adapters.artifacts.local import LocalStorageAdapter, LocalStorageBootstrap +from app.core.config import Settings +from app.core.hashing import canonical_json_hash +from app.modules.actors.models import ActorIdentityLink, ActorProfile +from app.modules.actors.service_identities import ServiceIdentity +from app.modules.artifacts.models import ( + ArtifactAdmissionCharge, + ArtifactAdmissionScope, + ArtifactContent, + ArtifactOperationReceipt, + ArtifactPutAttempt, + ArtifactPutAttemptCharge, + ArtifactReplica, + ArtifactStorageNamespace, + ArtifactUploadItem, + ArtifactUploadSession, +) +from app.modules.artifacts.repository import CheckerOutputAdmissionFacts +from app.modules.artifacts.schemas import ( + CheckerOutputArtifactAdmissionRequest, + ContributorArtifactAdmissionRequest, + GuideArtifactAdmissionRequest, +) +from app.modules.artifacts.service import ( + ArtifactAdmissionCapacityError, + ArtifactAdmissionConflictError, + ArtifactAdmissionRelationshipError, + ArtifactAdmissionService, + ArtifactStorageNamespaceSpec, + artifact_storage_namespace_spec, +) +from app.modules.authorization.runtime import ( + ActorKind, + ActorStatus, + AuthorizationContext, + IdentityLinkStatus, +) +from app.modules.projects.models import ( + GuideSourceSnapshot, + GuideSourceSnapshotItem, + Project, + ProjectGuide, +) +from app.modules.tasks.models import WorkstreamTask +from tests.artifact_store_helpers import ( + artifact_admission_limit_settings, + minted_source, +) + + +def _alembic_config() -> Config: + root = Path(__file__).resolve().parents[1] + config = Config(str(root / "alembic.ini")) + config.set_main_option("script_location", str(root / "alembic")) + return config + + +@pytest.fixture +def admission_database_env( + isolated_database_env: str, + migration_lock, +) -> str: + """Provide the exact head schema and remove all test evidence afterward.""" + config = _alembic_config() + with migration_lock(): + asyncio.run(_reset_admission_test_schema(isolated_database_env)) + command.upgrade(config, "head") + try: + yield isolated_database_env + finally: + asyncio.run(_reset_admission_test_schema(isolated_database_env)) + + +async def _reset_admission_test_schema(database_url: str) -> None: + engine = create_async_engine(database_url) + try: + async with engine.begin() as connection: + await connection.execute(text("drop schema if exists public cascade")) + await connection.execute(text("create schema public")) + finally: + await engine.dispose() + + +def _settings(tmp_path: Path, *, maximum_bytes: int = 1024) -> Settings: + durable_root = tmp_path / "durable" + durable_root.mkdir(mode=0o700, parents=True) + return Settings( + **artifact_admission_limit_settings(maximum_bytes), + environment="test", + artifact_store_backend="local", + artifact_local_root=durable_root, + artifact_scratch_root=tmp_path / "scratch", + artifact_scratch_minimum_free_bytes=0, + ) + + +def _namespace(settings: Settings) -> ArtifactStorageNamespaceSpec: + assert settings.artifact_local_root is not None + bootstrap = LocalStorageBootstrap( + LocalStorageAdapter(root=settings.artifact_local_root) + ) + try: + return artifact_storage_namespace_spec(settings, bootstrap) + finally: + bootstrap.close() + + +def _context( + *, + actor_profile_id: UUID | None = None, + identity_link_id: UUID | None = None, + actor_kind: ActorKind = ActorKind.HUMAN, +) -> AuthorizationContext: + return AuthorizationContext( + actor_profile_id=actor_profile_id or uuid4(), + actor_kind=actor_kind, + actor_status=ActorStatus.ACTIVE, + identity_link_id=identity_link_id or uuid4(), + identity_link_status=IdentityLinkStatus.ACTIVE, + request_id=uuid4(), + correlation_id=uuid4(), + ) + + +async def _seed_guide( + session, + *, + content_hash: str, + media_type: str, +) -> tuple[str, str]: + project_id = str(uuid4()) + guide_id = str(uuid4()) + snapshot_id = str(uuid4()) + item_id = str(uuid4()) + session.add( + Project( + id=project_id, + name="Admission project", + slug=f"admission-{project_id}", + ) + ) + await session.flush() + session.add( + ProjectGuide( + id=guide_id, + project_id=project_id, + version="v1", + status="draft", + content_markdown="# Guide", + created_by="test", + ) + ) + await session.flush() + session.add( + GuideSourceSnapshot( + id=snapshot_id, + project_id=project_id, + guide_id=guide_id, + guide_version="v1", + manifest_schema_version="v1", + manifest_json={"items": [item_id]}, + bundle_hash=canonical_json_hash({"items": [item_id]}), + captured_by="test", + ) + ) + await session.flush() + session.add( + GuideSourceSnapshotItem( + id=item_id, + source_snapshot_id=snapshot_id, + item_order=0, + source_kind="inline", + durable_ref="guide.md", + ingestion_adapter="inline", + content_hash=content_hash, + media_type=media_type, + ) + ) + await session.commit() + return project_id, item_id + + +async def _seed_contributor_items( + session, + *, + actor_profile_id: str, + commitments: tuple[tuple[str, int, str], ...], +) -> tuple[str, str, tuple[str, ...]]: + project_id = str(uuid4()) + task_id = str(uuid4()) + upload_session_id = str(uuid4()) + session.add( + Project( + id=project_id, + name="Contributor project", + slug=f"contributor-{project_id}", + ) + ) + await session.flush() + session.add( + WorkstreamTask( + id=task_id, + project_id=project_id, + title="Admission task", + description="Prove artifact admission.", + status="draft", + created_by="test", + ) + ) + await session.flush() + total_bytes = sum(byte_count for _, byte_count, _ in commitments) + session.add( + ArtifactUploadSession( + id=upload_session_id, + actor_id=actor_profile_id, + project_id=project_id, + task_id=task_id, + permitted_roles=["submission"], + state="open", + maximum_bytes=max(total_bytes, 1), + current_bytes=0, + reserved_bytes=total_bytes, + maximum_items=len(commitments), + current_items=0, + reserved_items=len(commitments), + expires_at=datetime.now(UTC) + timedelta(minutes=10), + cas_version=0, + ) + ) + await session.flush() + item_ids = [] + for index, (sha256, byte_count, media_type) in enumerate(commitments): + item_id = str(uuid4()) + item_ids.append(item_id) + session.add( + ArtifactUploadItem( + id=item_id, + session_id=upload_session_id, + logical_role=f"submission-{index}", + display_name=f"result-{index}.bin", + media_type=media_type, + reserved_bytes=byte_count, + expected_sha256=sha256, + expected_size=byte_count, + idempotency_key=f"put-{item_id}", + request_digest=canonical_json_hash( + { + "sha256": sha256, + "byte_count": byte_count, + "media_type": media_type, + } + ), + state="reserved", + cas_version=0, + ) + ) + await session.commit() + return project_id, task_id, tuple(item_ids) + + +async def _count(session, model: type) -> int: + value = await session.scalar(select(func.count()).select_from(model)) + assert value is not None + return value + + +async def test_guide_admission_derives_three_scopes_without_provider_evidence( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path) + namespace = _namespace(settings) + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + context = _context() + async with minted_source( + tmp_path / "scratch-source", + b"guide", + media_type="text/markdown", + ) as source: + project_id, item_id = await _seed_guide( + session, + content_hash=source.commitment.sha256, + media_type=source.commitment.media_type, + ) + result = await ArtifactAdmissionService( + session, + settings, + namespace, + ).admit( + GuideArtifactAdmissionRequest( + authorization_context=context, + guide_source_item_id=UUID(item_id), + source=source, + ) + ) + + async with minted_source( + tmp_path / "wrong-source", + b"different guide bytes", + media_type="text/markdown", + ) as wrong_source: + with pytest.raises( + ArtifactAdmissionRelationshipError, + match="guide source item relationship is unavailable", + ): + await ArtifactAdmissionService( + session, + settings, + namespace, + ).admit( + GuideArtifactAdmissionRequest( + authorization_context=context, + guide_source_item_id=UUID(item_id), + source=wrong_source, + ) + ) + + attempt = await session.get(ArtifactPutAttempt, str(result.attempt_id)) + scopes = ( + await session.execute( + select(ArtifactAdmissionScope).order_by( + ArtifactAdmissionScope.scope_type + ) + ) + ).scalars().all() + assert attempt is not None + assert attempt.status == "prepared" + assert attempt.project_id == project_id + assert attempt.task_id is None + assert attempt.executor_id is None + assert attempt.lease_expires_at is None + assert attempt.execution_generation == 0 + assert {scope.scope_type for scope in scopes} == { + "deployment", + "producer", + "project", + } + assert len(result.charge_ids) == 3 + assert await _count(session, ArtifactPutAttempt) == 1 + assert await _count(session, ArtifactContent) == 0 + assert await _count(session, ArtifactReplica) == 0 + assert await _count(session, ArtifactOperationReceipt) == 0 + finally: + await engine.dispose() + + +async def test_exact_replay_returns_one_attempt_and_one_charge_set( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path) + namespace = _namespace(settings) + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + async with minted_source(tmp_path / "scratch-source", b"same") as source: + _, _, item_ids = await _seed_contributor_items( + session, + actor_profile_id=str(context.actor_profile_id), + commitments=( + ( + source.commitment.sha256, + source.commitment.byte_count, + source.commitment.media_type, + ), + ), + ) + request = ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[0]), + source=source, + ) + first = await ArtifactAdmissionService( + session, + settings, + namespace, + ).admit(request) + replay = await ArtifactAdmissionService( + session, + settings, + namespace, + ).admit(request) + + assert replay.replayed is True + assert replay.attempt_id == first.attempt_id + assert replay.charge_ids == first.charge_ids + assert await _count(session, ArtifactPutAttempt) == 1 + assert await _count(session, ArtifactAdmissionCharge) == 4 + assert await _count(session, ArtifactPutAttemptCharge) == 4 + finally: + await engine.dispose() + + +async def test_same_content_distinct_operations_deduplicate_scope_bytes( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path) + namespace = _namespace(settings) + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with minted_source(tmp_path / "scratch-source", b"same") as source: + async with factory() as seed_session: + commitment = ( + source.commitment.sha256, + source.commitment.byte_count, + source.commitment.media_type, + ) + _, _, item_ids = await _seed_contributor_items( + seed_session, + actor_profile_id=str(context.actor_profile_id), + commitments=(commitment, commitment), + ) + + async def admit(item_id: str): + async with factory() as session: + return await ArtifactAdmissionService( + session, + settings, + namespace, + ).admit( + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_id), + source=source, + ) + ) + + results = await asyncio.gather(*(admit(item_id) for item_id in item_ids)) + + async with factory() as session: + assert all(result.replayed is False for result in results) + counters = ( + await session.execute(select(ArtifactAdmissionScope)) + ).scalars().all() + assert {counter.counted_bytes for counter in counters} == {4} + assert await _count(session, ArtifactAdmissionCharge) == 4 + assert await _count(session, ArtifactPutAttempt) == 2 + assert await _count(session, ArtifactPutAttemptCharge) == 8 + finally: + await engine.dispose() + + +async def test_completed_charge_deduplicates_and_released_charge_is_reacquired( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path) + namespace = _namespace(settings) + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with minted_source(tmp_path / "scratch-source", b"same") as source: + async with factory() as session: + commitment = ( + source.commitment.sha256, + source.commitment.byte_count, + source.commitment.media_type, + ) + _, _, item_ids = await _seed_contributor_items( + session, + actor_profile_id=str(context.actor_profile_id), + commitments=(commitment, commitment, commitment), + ) + service = ArtifactAdmissionService(session, settings, namespace) + await service.admit( + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[0]), + source=source, + ) + ) + + charges = ( + await session.execute(select(ArtifactAdmissionCharge)) + ).scalars().all() + completed_at = datetime.now(UTC) + for charge in charges: + charge.state = "completed" + charge.completed_at = completed_at + await session.commit() + + await service.admit( + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[1]), + source=source, + ) + ) + counters = ( + await session.execute(select(ArtifactAdmissionScope)) + ).scalars().all() + assert {counter.counted_bytes for counter in counters} == {4} + + released_at = datetime.now(UTC) + for charge in charges: + charge.state = "released" + charge.completed_at = None + charge.released_at = released_at + for counter in counters: + counter.counted_bytes = 0 + counter.cas_version += 1 + await session.commit() + + await service.admit( + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[2]), + source=source, + ) + ) + + refreshed_charges = ( + await session.execute(select(ArtifactAdmissionCharge)) + ).scalars().all() + refreshed_counters = ( + await session.execute(select(ArtifactAdmissionScope)) + ).scalars().all() + assert {charge.state for charge in refreshed_charges} == {"provisional"} + assert {charge.released_at for charge in refreshed_charges} == {None} + assert {charge.cas_version for charge in refreshed_charges} == {1} + assert {counter.counted_bytes for counter in refreshed_counters} == {4} + assert await _count(session, ArtifactAdmissionCharge) == 4 + assert await _count(session, ArtifactPutAttempt) == 3 + assert await _count(session, ArtifactPutAttemptCharge) == 12 + finally: + await engine.dispose() + + +async def test_concurrent_distinct_content_cannot_oversubscribe_any_scope( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path, maximum_bytes=6) + namespace = _namespace(settings) + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with minted_source(tmp_path / "scratch-a", b"aaaa") as first_source: + async with minted_source(tmp_path / "scratch-b", b"bbbb") as second_source: + async with factory() as seed_session: + _, _, item_ids = await _seed_contributor_items( + seed_session, + actor_profile_id=str(context.actor_profile_id), + commitments=( + ( + first_source.commitment.sha256, + first_source.commitment.byte_count, + first_source.commitment.media_type, + ), + ( + second_source.commitment.sha256, + second_source.commitment.byte_count, + second_source.commitment.media_type, + ), + ), + ) + + async def admit(item_id: str, source): + async with factory() as session: + return await ArtifactAdmissionService( + session, + settings, + namespace, + ).admit( + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_id), + source=source, + ) + ) + + outcomes = await asyncio.gather( + admit(item_ids[0], first_source), + admit(item_ids[1], second_source), + return_exceptions=True, + ) + + assert sum(not isinstance(value, BaseException) for value in outcomes) == 1 + assert sum( + isinstance(value, ArtifactAdmissionCapacityError) for value in outcomes + ) == 1 + async with factory() as session: + counters = ( + await session.execute(select(ArtifactAdmissionScope)) + ).scalars().all() + assert {counter.counted_bytes for counter in counters} == {4} + assert await _count(session, ArtifactPutAttempt) == 1 + assert await _count(session, ArtifactAdmissionCharge) == 4 + finally: + await engine.dispose() + + +async def test_capacity_failure_rolls_back_namespace_scopes_charges_and_attempt( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path, maximum_bytes=3) + namespace = _namespace(settings) + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + async with minted_source(tmp_path / "scratch-source", b"four") as source: + _, _, item_ids = await _seed_contributor_items( + session, + actor_profile_id=str(context.actor_profile_id), + commitments=( + ( + source.commitment.sha256, + source.commitment.byte_count, + source.commitment.media_type, + ), + ), + ) + with pytest.raises(ArtifactAdmissionCapacityError): + await ArtifactAdmissionService( + session, + settings, + namespace, + ).admit( + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[0]), + source=source, + ) + ) + + assert await _count(session, ArtifactStorageNamespace) == 0 + assert await _count(session, ArtifactAdmissionScope) == 0 + assert await _count(session, ArtifactAdmissionCharge) == 0 + assert await _count(session, ArtifactPutAttempt) == 0 + finally: + await engine.dispose() + + +async def test_changed_input_for_existing_operation_fails_closed( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path) + namespace = _namespace(settings) + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + async with minted_source(tmp_path / "scratch-a", b"aaaa") as first_source: + _, _, item_ids = await _seed_contributor_items( + session, + actor_profile_id=str(context.actor_profile_id), + commitments=( + ( + first_source.commitment.sha256, + first_source.commitment.byte_count, + first_source.commitment.media_type, + ), + ), + ) + await ArtifactAdmissionService(session, settings, namespace).admit( + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[0]), + source=first_source, + ) + ) + + async with minted_source(tmp_path / "scratch-b", b"bbbb") as changed_source: + item = await session.get(ArtifactUploadItem, item_ids[0]) + assert item is not None + item.expected_sha256 = changed_source.commitment.sha256 + item.expected_size = changed_source.commitment.byte_count + await session.commit() + with pytest.raises(ArtifactAdmissionConflictError): + await ArtifactAdmissionService(session, settings, namespace).admit( + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[0]), + source=changed_source, + ) + ) + finally: + await engine.dispose() + + +async def test_checker_output_requires_exact_active_fixed_service_identity( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path) + namespace = _namespace(settings) + actor_id = uuid4() + link_id = uuid4() + context = _context( + actor_profile_id=actor_id, + identity_link_id=link_id, + actor_kind=ActorKind.SERVICE, + ) + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + session.add( + ActorProfile( + id=str(actor_id), + actor_kind="service", + status="active", + provisioning_method="manual_service_provisioning", + service_identity=ServiceIdentity.ARTIFACT_CHECKER_OUTPUT.value, + created_by="test", + ) + ) + session.add( + ActorIdentityLink( + id=str(link_id), + actor_profile_id=str(actor_id), + issuer="https://issuer.example.test", + subject="checker-output-service", + subject_kind="service", + status="active", + linked_by="test", + ) + ) + await session.commit() + async with minted_source(tmp_path / "scratch-source", b"checker") as source: + service = ArtifactAdmissionService(session, settings, namespace) + service._repo.get_checker_output_admission_facts = AsyncMock( + return_value=CheckerOutputAdmissionFacts( + checker_run_id="run-id", + project_id="project-id", + task_id="task-id", + ) + ) + facts = await service._checker_output_facts( + CheckerOutputArtifactAdmissionRequest( + authorization_context=context, + checker_run_id=uuid4(), + logical_role="platform-review", + source=source, + ) + ) + assert facts.producer_ref == ( + ServiceIdentity.ARTIFACT_CHECKER_OUTPUT.value + ) + + forged = context.model_copy(update={"identity_link_id": uuid4()}) + with pytest.raises( + ArtifactAdmissionRelationshipError, + match="service identity is unavailable", + ): + await service._checker_output_facts( + CheckerOutputArtifactAdmissionRequest( + authorization_context=forged, + checker_run_id=uuid4(), + logical_role="platform-review", + source=source, + ) + ) + finally: + await engine.dispose() + + +def test_artifact_admission_migration_preserves_prior_rows_and_round_trips_empty( + isolated_database_env: str, + migration_lock, +) -> None: + config = _alembic_config() + namespace_fingerprint = "sha256:" + "a" * 64 + + async def seed_prior_namespace() -> None: + engine = create_async_engine(isolated_database_env) + try: + async with engine.begin() as connection: + await connection.execute( + text( + "insert into artifact_storage_namespaces " + "(id,backend,adapter,provider_profile,namespace_descriptor," + "namespace_fingerprint) values " + "('primary','local','local','local-v2','{}',:fingerprint)" + ), + {"fingerprint": namespace_fingerprint}, + ) + finally: + await engine.dispose() + + async def state() -> tuple[int, bool]: + engine = create_async_engine(isolated_database_env) + try: + async with engine.connect() as connection: + count = await connection.scalar( + text("select count(*) from artifact_storage_namespaces") + ) + table_exists = await connection.scalar( + text("select to_regclass('artifact_put_attempts') is not null") + ) + return int(count or 0), bool(table_exists) + finally: + await engine.dispose() + + async def cleanup() -> None: + engine = create_async_engine(isolated_database_env) + try: + async with engine.begin() as connection: + await connection.execute( + text("truncate table artifact_storage_namespaces cascade") + ) + finally: + await engine.dispose() + + with migration_lock(): + try: + asyncio.run(_reset_admission_test_schema(isolated_database_env)) + command.upgrade(config, "0026_actor_profile_lifecycle") + asyncio.run(seed_prior_namespace()) + command.upgrade(config, "0027_artifact_admission") + assert asyncio.run(state()) == (1, True) + command.downgrade(config, "0026_actor_profile_lifecycle") + assert asyncio.run(state()) == (1, False) + command.upgrade(config, "0027_artifact_admission") + assert asyncio.run(state()) == (1, True) + asyncio.run(cleanup()) + finally: + asyncio.run(_reset_admission_test_schema(isolated_database_env)) + + +def test_artifact_admission_migration_refuses_populated_downgrade( + isolated_database_env: str, + migration_lock, +) -> None: + config = _alembic_config() + + async def seed_scope() -> None: + engine = create_async_engine(isolated_database_env) + try: + async with engine.begin() as connection: + await connection.execute( + text( + "insert into artifact_admission_scopes " + "(scope_type,scope_id,limit_bytes,counted_bytes,cas_version) " + "values ('deployment','primary',10,0,0)" + ) + ) + finally: + await engine.dispose() + + async def cleanup() -> None: + engine = create_async_engine(isolated_database_env) + try: + async with engine.begin() as connection: + await connection.execute( + text("truncate table artifact_admission_scopes cascade") + ) + finally: + await engine.dispose() + + with migration_lock(): + try: + asyncio.run(_reset_admission_test_schema(isolated_database_env)) + command.upgrade(config, "0027_artifact_admission") + asyncio.run(seed_scope()) + with pytest.raises( + RuntimeError, + match="cannot downgrade populated artifact admission ledger", + ): + command.downgrade(config, "0026_actor_profile_lifecycle") + asyncio.run(cleanup()) + command.downgrade(config, "0026_actor_profile_lifecycle") + finally: + asyncio.run(_reset_admission_test_schema(isolated_database_env)) diff --git a/backend/tests/test_artifact_architecture.py b/backend/tests/test_artifact_architecture.py index 65aed7da1..d1ba72b9e 100644 --- a/backend/tests/test_artifact_architecture.py +++ b/backend/tests/test_artifact_architecture.py @@ -35,6 +35,13 @@ "ArtifactBindingResourceType", } RAW_TYPES = {"ArtifactStore", "ArtifactStorageOrchestrator"} +INTERNAL_ADMISSION_TYPES = { + "ArtifactAdmissionService", + "ArtifactAdmissionResult", + "CheckerOutputArtifactAdmissionRequest", + "ContributorArtifactAdmissionRequest", + "GuideArtifactAdmissionRequest", +} PROVIDER_METHODS = {"put", "observe_put_result", "open", "head"} CONCRETE_ADAPTER_MODULES = { "app.adapters.artifacts.local", @@ -76,7 +83,7 @@ def test_product_api_and_workers_cannot_import_or_inject_raw_artifact_types() -> for node in ast.walk(tree): if isinstance(node, ast.ImportFrom): imported = {alias.name for alias in node.names} - forbidden = imported & RAW_TYPES + forbidden = imported & (RAW_TYPES | INTERNAL_ADMISSION_TYPES) if forbidden: violations.append(f"{path.relative_to(BACKEND_ROOT)} imports {sorted(forbidden)}") if ( @@ -104,6 +111,22 @@ def test_product_api_and_workers_cannot_import_or_inject_raw_artifact_types() -> assert violations == [] +def test_artifact_domain_has_no_provider_execution_during_admission_foundation() -> None: + """Keep 02C1 free of all provider write and acknowledgement execution.""" + violations: list[str] = [] + for path in _python_files(APP_ROOT / "modules" / "artifacts"): + for node in ast.walk(_tree(path)): + if ( + isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) + and node.func.attr in {"put", "observe_put_result"} + ): + violations.append( + f"{path.relative_to(BACKEND_ROOT)} calls {node.func.attr}" + ) + assert violations == [] + + def test_concrete_adapter_construction_has_one_composition_path() -> None: factory_calls: list[Path] = [] adapter_calls: list[Path] = [] diff --git a/backend/tests/test_artifact_cleanup_wiring.py b/backend/tests/test_artifact_cleanup_wiring.py index 1dd1c501f..20535a527 100644 --- a/backend/tests/test_artifact_cleanup_wiring.py +++ b/backend/tests/test_artifact_cleanup_wiring.py @@ -12,10 +12,12 @@ from app.core.config import Settings, get_settings from app.main import create_app from app.interfaces.artifacts import ArtifactProviderLiveProofRequiredError +from tests.artifact_store_helpers import artifact_admission_limit_settings def _enabled_settings(tmp_path: Path, **changes: object) -> Settings: values: dict[str, object] = { + **artifact_admission_limit_settings(), "environment": "test", "artifact_store_backend": "local", "artifact_local_root": tmp_path / "store", @@ -81,6 +83,7 @@ async def test_production_auth_validation_precedes_artifact_startup( events: list[str] = [] settings = Settings( + **artifact_admission_limit_settings(), environment="production", artifact_store_backend="s3_compatible", artifact_scratch_root=tmp_path / "scratch", @@ -164,6 +167,7 @@ async def test_aws_s3_startup_requires_live_provider_proof( monkeypatch.setattr(main_module, "build_auth_verifier", lambda _settings: None) app = create_app( Settings( + **artifact_admission_limit_settings(), environment="production", artifact_store_backend="s3_compatible", artifact_scratch_root=tmp_path / "scratch", @@ -351,6 +355,7 @@ def test_aws_s3_is_runtime_ineligible_for_celery_and_cleanup_task( """Apply the same inactive-provider guard to API and worker entry points.""" celery_module, artifacts_module = _load_worker_modules(monkeypatch, request) settings = Settings( + **artifact_admission_limit_settings(), environment="production", celery_task_always_eager=True, artifact_store_backend="s3_compatible", diff --git a/backend/tests/test_artifacts.py b/backend/tests/test_artifacts.py index 47a2086af..67ccfcaee 100644 --- a/backend/tests/test_artifacts.py +++ b/backend/tests/test_artifacts.py @@ -1,1107 +1,108 @@ -"""PostgreSQL integration tests for the ArtifactStore v2 orchestration boundary.""" +"""Focused tests for artifact namespace fencing after direct-write removal.""" from __future__ import annotations -import asyncio -from collections.abc import AsyncIterator -from datetime import UTC, datetime, timedelta from pathlib import Path -from unittest.mock import AsyncMock -from uuid import uuid4 -from alembic import command -from alembic.config import Config import pytest -from sqlalchemy import select, text -from sqlalchemy.exc import IntegrityError -from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine from app.adapters.artifacts.local import LocalStorageAdapter, LocalStorageBootstrap from app.core.config import Settings -from app.core.hashing import canonical_json_hash -from app.interfaces.artifacts import ( - ArtifactByteRange, - ArtifactConfigurationError, - ArtifactInputMismatchError, - ArtifactIntegrityError, - ArtifactObjectHead, - ArtifactPutObservation, - ArtifactPutResult, - ArtifactStoreUnavailableError, - ArtifactStoreNamespaceIdentity, -) from app.interfaces.external_services import ExternalServiceAdapterIdentity from app.modules.artifacts.models import ( - ArtifactContent, ArtifactOperationReceipt, ArtifactReplica, ArtifactStorageNamespace, - ArtifactUploadItem, - ArtifactUploadSession, ) from app.modules.artifacts.service import ( - ArtifactIngestStateError, ArtifactStorageNamespaceError, - ArtifactStorageNamespaceSpec, - ArtifactStorageOrchestrator, - ProviderAttemptFence, artifact_storage_namespace_spec, - validate_artifact_storage_namespace_at_startup, -) -import app.modules.artifacts.service as artifact_service_module -from app.modules.artifacts.sources import ArtifactCommitment, CommittedArtifactSource -from app.modules.projects.models import Project -from tests.artifact_store_helpers import ( - local_namespace_claim, - minted_source as _minted_source, ) - - -def _alembic_config() -> Config: - root = Path(__file__).resolve().parents[1] - config = Config(str(root / "alembic.ini")) - config.set_main_option("script_location", str(root / "alembic")) - return config - - -@pytest.fixture -def artifact_database_env(isolated_database_env: str, migration_lock) -> str: - """Provide one fully migrated empty PostgreSQL database per test.""" - config = _alembic_config() - with migration_lock(): - asyncio.run(_truncate_artifacts_if_present(isolated_database_env)) - command.downgrade(config, "base") - command.upgrade(config, "head") - yield isolated_database_env - with migration_lock(): - asyncio.run(_truncate_artifacts(isolated_database_env)) - command.downgrade(config, "base") - - -async def _truncate_artifacts(database_url: str) -> None: - engine = create_async_engine(database_url) - try: - async with engine.begin() as connection: - await connection.execute( - text( - "truncate table artifact_storage_namespaces, artifact_upload_sessions, " - "artifact_contents cascade" - ) - ) - finally: - await engine.dispose() - - -async def _truncate_artifacts_if_present(database_url: str) -> None: - """Clear only artifact tables that exist at the database's current revision.""" - engine = create_async_engine(database_url) - candidates = ( - "artifact_storage_namespaces", - "artifact_upload_sessions", - "artifact_contents", - ) - try: - async with engine.begin() as connection: - existing = [ - table_name - for table_name in candidates - if await connection.scalar( - text("select to_regclass(:table_name) is not null"), - {"table_name": f"public.{table_name}"}, - ) - ] - if existing: - await connection.execute(text(f"truncate table {', '.join(existing)} cascade")) - finally: - await engine.dispose() +from tests.artifact_store_helpers import artifact_admission_limit_settings def _settings(tmp_path: Path) -> Settings: - (tmp_path / "store").mkdir(mode=0o700, exist_ok=True) + root = tmp_path / "store" + root.mkdir(mode=0o700, parents=True) return Settings( + **artifact_admission_limit_settings(), environment="test", artifact_store_backend="local", - artifact_local_root=tmp_path / "store", + artifact_local_root=root, artifact_scratch_root=tmp_path / "scratch", artifact_scratch_minimum_free_bytes=0, ) -async def _seed_reserved_item( - session, - commitment: ArtifactCommitment, -) -> str: - project_id = str(uuid4()) - session_id = str(uuid4()) - item_id = str(uuid4()) - session.add(Project(id=project_id, name="Artifact project", slug=f"artifact-{project_id}")) - await session.flush() - session.add( - ArtifactUploadSession( - id=session_id, - actor_id="actor-1", - project_id=project_id, - permitted_roles=["submission"], - state="open", - maximum_bytes=1024, - current_bytes=0, - reserved_bytes=commitment.byte_count, - maximum_items=1, - current_items=0, - reserved_items=1, - expires_at=datetime.now(UTC) + timedelta(minutes=5), - cas_version=0, - ) - ) - await session.flush() - session.add( - ArtifactUploadItem( - id=item_id, - session_id=session_id, - logical_role="submission", - display_name="result.bin", - media_type=commitment.media_type, - reserved_bytes=commitment.byte_count, - expected_sha256=commitment.sha256, - expected_size=commitment.byte_count, - idempotency_key=f"put-{item_id}", - request_digest=canonical_json_hash( - { - "sha256": commitment.sha256, - "byte_count": commitment.byte_count, - "media_type": commitment.media_type, - } - ), - state="reserved", - cas_version=0, - ) - ) - await session.commit() - return item_id - - -async def _seed_reserved_items_in_one_session( - session, - commitments: tuple[ArtifactCommitment, ...], -) -> tuple[str, ...]: - """Create multiple independently fenced items under one aggregate ledger.""" - project_id = str(uuid4()) - session_id = str(uuid4()) - item_ids = tuple(str(uuid4()) for _ in commitments) - total_bytes = sum(commitment.byte_count for commitment in commitments) - session.add(Project(id=project_id, name="Artifact project", slug=f"artifact-{project_id}")) - await session.flush() - session.add( - ArtifactUploadSession( - id=session_id, - actor_id="actor-1", - project_id=project_id, - permitted_roles=["submission"], - state="open", - maximum_bytes=1024, - current_bytes=0, - reserved_bytes=total_bytes, - maximum_items=len(commitments), - current_items=0, - reserved_items=len(commitments), - expires_at=datetime.now(UTC) + timedelta(minutes=5), - cas_version=0, +def _bootstrap(settings: Settings) -> LocalStorageBootstrap: + assert settings.artifact_local_root is not None + return LocalStorageBootstrap( + LocalStorageAdapter( + root=settings.artifact_local_root, + buffer_bytes=settings.artifact_stream_buffer_bytes, + lock_timeout_seconds=settings.artifact_operation_lock_timeout_seconds, ) ) - await session.flush() - for item_id, commitment in zip(item_ids, commitments, strict=True): - session.add( - ArtifactUploadItem( - id=item_id, - session_id=session_id, - logical_role="submission", - display_name=f"{item_id}.bin", - media_type=commitment.media_type, - reserved_bytes=commitment.byte_count, - expected_sha256=commitment.sha256, - expected_size=commitment.byte_count, - idempotency_key=f"put-{item_id}", - request_digest=canonical_json_hash( - { - "sha256": commitment.sha256, - "byte_count": commitment.byte_count, - "media_type": commitment.media_type, - } - ), - state="reserved", - cas_version=0, - ) - ) - await session.commit() - return item_ids -@pytest.mark.asyncio -async def test_put_acknowledgement_stops_at_pending_verification( - artifact_database_env: str, +def test_namespace_descriptor_is_canonical_and_excludes_local_path( tmp_path: Path, ) -> None: - content = b"artifact-v2" - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) settings = _settings(tmp_path) - bootstrap = LocalStorageBootstrap(LocalStorageAdapter(root=tmp_path / "store", buffer_bytes=2)) - namespace = artifact_storage_namespace_spec(settings, bootstrap) - store = bootstrap.initialize_after_namespace_claim(local_namespace_claim(bootstrap)) + bootstrap = _bootstrap(settings) try: - async with _minted_source(tmp_path / "scratch", content) as source: - async with factory() as session: - item_id = await _seed_reserved_item(session, source.commitment) - result = await ArtifactStorageOrchestrator( - session, store, namespace - ).put_reserved_item(item_id, source) - async with _minted_source(tmp_path / "scratch-replay", content) as replay_source: - async with factory() as session: - replay_item_id = await _seed_reserved_item(session, replay_source.commitment) - replay = await ArtifactStorageOrchestrator( - session, store, namespace - ).put_reserved_item(replay_item_id, replay_source) - assert replay.replayed is True - - async with factory() as session: - item = await session.get(ArtifactUploadItem, item_id) - replica = (await session.execute(select(ArtifactReplica))).scalar_one() - receipts = (await session.execute(select(ArtifactOperationReceipt))).scalars().all() - receipt = next(value for value in receipts if value.upload_item_id == item_id) - replay_receipt = next( - value for value in receipts if value.upload_item_id == replay_item_id - ) - namespace_row = (await session.execute(select(ArtifactStorageNamespace))).scalar_one() - assert item is not None - assert item.state == "stored_pending_verification" - assert item.provider_object_ref == result.provider_object_ref - assert replica.verification_state == "pending" - assert replica.availability_state == "unknown" - assert replica.integrity_state == "unknown" - assert replica.namespace_fingerprint == namespace_row.namespace_fingerprint - assert receipt.operation == "put" - assert receipt.outcome == "stored_pending_verification" - assert receipt.provider_object_ref == result.provider_object_ref - assert receipt.replayed is False - assert len(receipts) == 2 - assert replay_receipt.replayed is True - assert await store.head(result.provider_object_ref) == ArtifactObjectHead( - result.provider_object_ref, - exists=True, - byte_count=len(content), - media_type=None, - ) - finally: - store.close() - await engine.dispose() - - -class _UnavailableStore: - """Count provider calls and fail with one sanitized retryable error.""" - - identity = ExternalServiceAdapterIdentity("artifact_store", "local") - namespace_identity = ArtifactStoreNamespaceIdentity( - provider_profile="local-v2", - descriptor_items=( - ("private_prefix", "objects/sha256"), - ("private_root_identity", "sha256:" + "0" * 64), - ), - ) - - def __init__(self) -> None: - self.put_calls = 0 - - async def put(self, _source: CommittedArtifactSource) -> ArtifactPutResult: - self.put_calls += 1 - raise ArtifactStoreUnavailableError() - - async def observe_put_result(self, _commitment: ArtifactCommitment) -> ArtifactPutObservation: - raise NotImplementedError - - def open( - self, _provider_object_ref: str, _byte_range: ArtifactByteRange | None = None - ) -> AsyncIterator[bytes]: - raise NotImplementedError - - async def head(self, _provider_object_ref: str) -> ArtifactObjectHead: - raise NotImplementedError - - -class _TerminalStore(_UnavailableStore): - async def put(self, _source: CommittedArtifactSource) -> ArtifactPutResult: - self.put_calls += 1 - raise ArtifactInputMismatchError() - - -class _AcknowledgementUnknownThenReplayStore(_UnavailableStore): - """Lose the first acknowledgement and return an exact replay next.""" - - async def put(self, source: CommittedArtifactSource) -> ArtifactPutResult: - self.put_calls += 1 - if self.put_calls == 1: - raise RuntimeError("provider acknowledgement lost") - digest_hex = source.commitment.sha256[7:] - return ArtifactPutResult( - f"sha256/{digest_hex[:2]}/{digest_hex[2:]}", - replayed=True, - ) - - -class _CancelledStore(_UnavailableStore): - async def put(self, _source: CommittedArtifactSource) -> ArtifactPutResult: - self.put_calls += 1 - raise asyncio.CancelledError("provider put cancelled") - - -class _AcknowledgingStore(_UnavailableStore): - async def put(self, source: CommittedArtifactSource) -> ArtifactPutResult: - self.put_calls += 1 - digest_hex = source.commitment.sha256[7:] - return ArtifactPutResult( - f"sha256/{digest_hex[:2]}/{digest_hex[2:]}", - replayed=False, - ) - - -class _NamespaceObservingAcknowledgingStore(_AcknowledgingStore): - """Assert the committed namespace exists before the provider is invoked.""" - - def __init__(self, factory: async_sessionmaker[AsyncSession]) -> None: - super().__init__() - self._factory = factory - self.observed_namespace_fingerprint: str | None = None - - async def put(self, source: CommittedArtifactSource) -> ArtifactPutResult: - async with self._factory() as session: - namespace = await session.scalar(select(ArtifactStorageNamespace)) - assert namespace is not None - self.observed_namespace_fingerprint = namespace.namespace_fingerprint - return await super().put(source) - - -class _ConcurrentAcknowledgingStore(_AcknowledgingStore): - """Release two same-object acknowledgements into finalization together.""" - - def __init__(self) -> None: - super().__init__() - self._arrivals = 0 - self._release = asyncio.Event() - - async def put(self, source: CommittedArtifactSource) -> ArtifactPutResult: - self.put_calls += 1 - self._arrivals += 1 - arrival = self._arrivals - if self._arrivals == 2: - self._release.set() - await asyncio.wait_for(self._release.wait(), timeout=2) - digest_hex = source.commitment.sha256[7:] - return ArtifactPutResult( - f"sha256/{digest_hex[:2]}/{digest_hex[2:]}", - replayed=arrival > 1, - ) - - -class _ConcurrentDistinctAcknowledgingStore(_ConcurrentAcknowledgingStore): - """Release two distinct initial publications into finalization together.""" - - async def put(self, source: CommittedArtifactSource) -> ArtifactPutResult: - result = await super().put(source) - return ArtifactPutResult(result.provider_object_ref, replayed=False) - - -@pytest.mark.asyncio -async def test_concurrent_same_object_finalization_reuses_one_replica( - artifact_database_env: str, - tmp_path: Path, -) -> None: - """Finalize two exact replays with one replica and two receipts.""" - content = b"concurrent-artifact-v2" - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - store = _ConcurrentAcknowledgingStore() - namespace = artifact_storage_namespace_spec(_settings(tmp_path), store) - try: - async with _minted_source(tmp_path / "scratch-first", content) as first_source: - async with _minted_source(tmp_path / "scratch-second", content) as second_source: - async with factory() as session: - first_item_id = await _seed_reserved_item(session, first_source.commitment) - second_item_id = await _seed_reserved_item(session, second_source.commitment) - await ArtifactStorageOrchestrator( - session, store, namespace - ).ensure_storage_namespace() - async with session.begin(): - session.add( - ArtifactContent( - id=str(uuid4()), - sha256=first_source.commitment.sha256, - byte_count=first_source.commitment.byte_count, - media_type=first_source.commitment.media_type, - normalized_display_name="preexisting.bin", - ) - ) - - async def finalize(item_id: str, source: CommittedArtifactSource) -> None: - async with factory() as session: - await ArtifactStorageOrchestrator( - session, store, namespace - ).put_reserved_item(item_id, source) - - await asyncio.gather( - finalize(first_item_id, first_source), - finalize(second_item_id, second_source), - ) - - async with factory() as session: - items = ( - ( - await session.execute( - select(ArtifactUploadItem).where( - ArtifactUploadItem.id.in_((first_item_id, second_item_id)) - ) - ) - ) - .scalars() - .all() - ) - replicas = (await session.execute(select(ArtifactReplica))).scalars().all() - receipts = (await session.execute(select(ArtifactOperationReceipt))).scalars().all() - - assert {item.state for item in items} == {"stored_pending_verification"} - assert len(items) == 2 - assert len(replicas) == 1 - assert replicas[0].verification_state == "pending" - assert replicas[0].availability_state == "unknown" - assert replicas[0].integrity_state == "unknown" - assert len(receipts) == 2 - assert {receipt.replica_id for receipt in receipts} == {replicas[0].id} - finally: - await engine.dispose() - - -@pytest.mark.asyncio -async def test_independent_items_in_one_session_finalize_without_shared_cas_replay( - artifact_database_env: str, - tmp_path: Path, -) -> None: - """Fence provider acknowledgements per item while accounting one session.""" - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - store = _ConcurrentDistinctAcknowledgingStore() - namespace = artifact_storage_namespace_spec(_settings(tmp_path), store) - try: - async with _minted_source(tmp_path / "scratch-first", b"first-item") as first: - async with _minted_source(tmp_path / "scratch-second", b"second-item") as second: - async with factory() as session: - item_ids = await _seed_reserved_items_in_one_session( - session, - (first.commitment, second.commitment), - ) - await ArtifactStorageOrchestrator( - session, store, namespace - ).ensure_storage_namespace() - - async def finalize(item_id: str, source: CommittedArtifactSource) -> None: - async with factory() as candidate_session: - await ArtifactStorageOrchestrator( - candidate_session, store, namespace - ).put_reserved_item(item_id, source) - - await asyncio.gather( - finalize(item_ids[0], first), - finalize(item_ids[1], second), - ) - - async with factory() as session: - items = ( - ( - await session.execute( - select(ArtifactUploadItem).where(ArtifactUploadItem.id.in_(item_ids)) - ) - ) - .scalars() - .all() - ) - upload_session = await session.get(ArtifactUploadSession, items[0].session_id) - receipts = (await session.execute(select(ArtifactOperationReceipt))).scalars().all() - - assert {item.state for item in items} == {"stored_pending_verification"} - assert upload_session is not None - assert upload_session.reserved_bytes == 0 - assert upload_session.reserved_items == 0 - assert upload_session.current_bytes == len(b"first-item") + len(b"second-item") - assert upload_session.current_items == 2 - assert len(receipts) == 2 - finally: - await engine.dispose() - - -@pytest.mark.asyncio -async def test_database_rejects_partial_upload_result_metadata( - artifact_database_env: str, - tmp_path: Path, -) -> None: - """Require both stored-result references or neither in every item state.""" - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - try: - async with _minted_source(tmp_path / "scratch", b"constraint") as source: - async with factory() as session: - item_id = await _seed_reserved_item(session, source.commitment) - content_id = str(uuid4()) - session.add( - ArtifactContent( - id=content_id, - sha256=source.commitment.sha256, - byte_count=source.commitment.byte_count, - media_type=source.commitment.media_type, - normalized_display_name="constraint.bin", - ) - ) - await session.commit() - - with pytest.raises(IntegrityError): - async with engine.begin() as connection: - await connection.execute( - text( - "update artifact_upload_items set content_id = :content_id " - "where id = :item_id" - ), - {"content_id": content_id, "item_id": item_id}, - ) - with pytest.raises(IntegrityError): - async with engine.begin() as connection: - await connection.execute( - text( - "update artifact_upload_items " - "set provider_object_ref = 'sha256/00/object' where id = :item_id" - ), - {"item_id": item_id}, - ) - - async with factory() as session: - item = await session.get(ArtifactUploadItem, item_id) - assert item is not None - assert item.content_id is None - assert item.provider_object_ref is None - finally: - await engine.dispose() - - -def test_reservation_accounting_rejects_drift_without_clamping() -> None: - """Never consume capacity belonging to a different upload item.""" - upload_session = ArtifactUploadSession(reserved_bytes=3, reserved_items=1) - item = ArtifactUploadItem(reserved_bytes=4) - - with pytest.raises(ArtifactIntegrityError, match="reservation accounting"): - ArtifactStorageOrchestrator._apply_committed_accounting(upload_session, item, 4) - - assert upload_session.reserved_bytes == 3 - assert upload_session.reserved_items == 1 - - -class _FinalizationCancelledOrchestrator(ArtifactStorageOrchestrator): - async def _finalize_put( - self, - item_id: str, - commitment: ArtifactCommitment, - result: ArtifactPutResult, - fence: ProviderAttemptFence, - *, - correlation_id: str, - ) -> None: - del item_id, commitment, result, fence, correlation_id - raise asyncio.CancelledError("put finalization cancelled") - - -async def _assert_replay_required_without_durable_facts( - factory: async_sessionmaker, - item_id: str, -) -> None: - async with factory() as session: - item = await session.get(ArtifactUploadItem, item_id) - assert item is not None - upload_session = await session.get(ArtifactUploadSession, item.session_id) - assert upload_session is not None - assert item.state == "replay_required" - assert item.error_code == "artifact_put_acknowledgement_unknown" - assert upload_session.reserved_bytes == item.reserved_bytes - assert upload_session.reserved_items == 1 - assert await session.scalar(select(ArtifactContent)) is None - assert await session.scalar(select(ArtifactReplica)) is None - assert await session.scalar(select(ArtifactOperationReceipt)) is None - - -@pytest.mark.asyncio -async def test_retryable_put_marks_existing_item_replay_required( - artifact_database_env: str, - tmp_path: Path, -) -> None: - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - store = _UnavailableStore() - try: - async with _minted_source(tmp_path / "scratch", b"retry") as source: - async with factory() as session: - item_id = await _seed_reserved_item(session, source.commitment) - orchestrator = ArtifactStorageOrchestrator( - session, - store, - artifact_storage_namespace_spec(_settings(tmp_path), store), - ) - with pytest.raises(ArtifactStoreUnavailableError): - await orchestrator.put_reserved_item(item_id, source) - - async with factory() as session: - item = await session.get(ArtifactUploadItem, item_id) - assert item is not None - assert item.state == "replay_required" - assert item.error_code == "artifact_put_acknowledgement_unknown" - assert await session.scalar(select(ArtifactReplica)) is None - assert await session.scalar(select(ArtifactOperationReceipt)) is None - assert store.put_calls == 1 + spec = artifact_storage_namespace_spec(settings, bootstrap) finally: - await engine.dispose() + bootstrap.close() - -@pytest.mark.asyncio -async def test_same_item_replay_finalizes_once_without_double_accounting( - artifact_database_env: str, - tmp_path: Path, -) -> None: - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - store = _AcknowledgementUnknownThenReplayStore() - content = b"same-item-replay" - try: - async with _minted_source(tmp_path / "scratch-first", content) as source: - async with factory() as session: - item_id = await _seed_reserved_item(session, source.commitment) - orchestrator = ArtifactStorageOrchestrator( - session, - store, - artifact_storage_namespace_spec(_settings(tmp_path), store), - ) - with pytest.raises(RuntimeError, match="acknowledgement lost"): - await orchestrator.put_reserved_item(item_id, source) - - async with _minted_source(tmp_path / "scratch-replay", content) as replay: - async with factory() as session: - result = await ArtifactStorageOrchestrator( - session, - store, - artifact_storage_namespace_spec(_settings(tmp_path), store), - ).put_reserved_item(item_id, replay) - - assert result.replayed is True - async with factory() as session: - item = await session.get(ArtifactUploadItem, item_id) - assert item is not None - upload_session = await session.get(ArtifactUploadSession, item.session_id) - assert upload_session is not None - receipts = (await session.execute(select(ArtifactOperationReceipt))).scalars().all() - replicas = (await session.execute(select(ArtifactReplica))).scalars().all() - contents = (await session.execute(select(ArtifactContent))).scalars().all() - - assert item.state == "stored_pending_verification" - assert item.error_code is None - assert upload_session.reserved_bytes == 0 - assert upload_session.reserved_items == 0 - assert upload_session.current_bytes == len(content) - assert upload_session.current_items == 1 - assert len(contents) == 1 - assert len(replicas) == 1 - assert len(receipts) == 1 - assert receipts[0].upload_item_id == item_id - assert receipts[0].replayed is True - assert store.put_calls == 2 - finally: - await engine.dispose() - - -@pytest.mark.asyncio -async def test_provider_cancellation_persists_replay_required_without_facts( - artifact_database_env: str, - tmp_path: Path, -) -> None: - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - store = _CancelledStore() - try: - async with _minted_source(tmp_path / "scratch", b"provider cancellation") as source: - async with factory() as session: - item_id = await _seed_reserved_item(session, source.commitment) - orchestrator = ArtifactStorageOrchestrator( - session, - store, - artifact_storage_namespace_spec(_settings(tmp_path), store), - ) - with pytest.raises(asyncio.CancelledError, match="provider put cancelled"): - await orchestrator.put_reserved_item(item_id, source) - - await _assert_replay_required_without_durable_facts(factory, item_id) - assert store.put_calls == 1 - finally: - await engine.dispose() + assert spec.backend == "local" + assert spec.adapter == "local" + assert spec.provider_profile == "local-v2" + assert spec.namespace_descriptor["provider_profile"] == "local-v2" + assert str(tmp_path) not in repr(spec.namespace_descriptor) + assert spec.namespace_fingerprint.startswith("sha256:") -@pytest.mark.asyncio -async def test_finalization_cancellation_persists_replay_required_without_facts( - artifact_database_env: str, - tmp_path: Path, -) -> None: - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - store = _AcknowledgingStore() +def test_namespace_fingerprint_changes_when_pinned_root_changes(tmp_path: Path) -> None: + first = _settings(tmp_path / "first") + second = _settings(tmp_path / "second") + first_bootstrap = _bootstrap(first) + second_bootstrap = _bootstrap(second) try: - async with _minted_source(tmp_path / "scratch", b"finalization cancellation") as source: - async with factory() as session: - item_id = await _seed_reserved_item(session, source.commitment) - orchestrator = _FinalizationCancelledOrchestrator( - session, - store, - artifact_storage_namespace_spec(_settings(tmp_path), store), - ) - with pytest.raises(asyncio.CancelledError, match="finalization cancelled"): - await orchestrator.put_reserved_item(item_id, source) - - await _assert_replay_required_without_durable_facts(factory, item_id) - assert store.put_calls == 1 + first_fingerprint = artifact_storage_namespace_spec( + first, + first_bootstrap, + ).namespace_fingerprint + second_fingerprint = artifact_storage_namespace_spec( + second, + second_bootstrap, + ).namespace_fingerprint finally: - await engine.dispose() - + first_bootstrap.close() + second_bootstrap.close() -@pytest.mark.asyncio -async def test_terminal_put_failure_releases_reservation_and_fails_item( - artifact_database_env: str, - tmp_path: Path, -) -> None: - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - store = _TerminalStore() - try: - async with _minted_source(tmp_path / "scratch", b"terminal") as source: - async with factory() as session: - item_id = await _seed_reserved_item(session, source.commitment) - with pytest.raises(ArtifactInputMismatchError): - await ArtifactStorageOrchestrator( - session, - store, - artifact_storage_namespace_spec(_settings(tmp_path), store), - ).put_reserved_item(item_id, source) + assert first_fingerprint != second_fingerprint - async with factory() as session: - item = await session.get(ArtifactUploadItem, item_id) - assert item is not None - upload_session = await session.get(ArtifactUploadSession, item.session_id) - assert item.state == "failed" - assert item.error_code == "artifact_input_mismatch" - assert upload_session is not None - assert upload_session.reserved_bytes == 0 - assert upload_session.reserved_items == 0 - assert store.put_calls == 1 - finally: - await engine.dispose() - -@pytest.mark.asyncio -async def test_changed_commitment_is_rejected_before_provider_io( - artifact_database_env: str, - tmp_path: Path, -) -> None: - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - store = _UnavailableStore() - try: - async with _minted_source(tmp_path / "scratch-first", b"first") as first: - async with factory() as session: - item_id = await _seed_reserved_item(session, first.commitment) - async with _minted_source(tmp_path / "scratch-second", b"second") as second: - async with factory() as session: - with pytest.raises(ArtifactIngestStateError, match="commitment changed"): - await ArtifactStorageOrchestrator( - session, - store, - artifact_storage_namespace_spec(_settings(tmp_path), store), - ).put_reserved_item(item_id, second) - assert store.put_calls == 0 - finally: - await engine.dispose() - - -@pytest.mark.asyncio -async def test_namespace_mismatch_fails_before_provider_io( - artifact_database_env: str, - tmp_path: Path, -) -> None: - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - store = _UnavailableStore() +def test_namespace_spec_rejects_adapter_identity_drift(tmp_path: Path) -> None: settings = _settings(tmp_path) - canonical = artifact_storage_namespace_spec(settings, store) - conflicting_descriptor = dict(canonical.namespace_descriptor) - conflicting_descriptor["private_root_identity"] = "sha256:" + "f" * 64 - conflicting = ArtifactStorageNamespaceSpec( - backend=canonical.backend, - adapter=canonical.adapter, - provider_profile=canonical.provider_profile, - namespace_descriptor=conflicting_descriptor, - namespace_fingerprint=canonical_json_hash(conflicting_descriptor), - ) - try: - async with factory() as session: - await ArtifactStorageOrchestrator(session, store, canonical).ensure_storage_namespace() - async with _minted_source(tmp_path / "scratch", b"blocked") as source: - async with factory() as session: - item_id = await _seed_reserved_item(session, source.commitment) - with pytest.raises(ArtifactStorageNamespaceError): - await ArtifactStorageOrchestrator( - session, store, conflicting - ).put_reserved_item(item_id, source) - assert store.put_calls == 0 - finally: - await engine.dispose() - -@pytest.mark.asyncio -async def test_concurrent_different_first_namespace_writers_have_one_winner( - artifact_database_env: str, - tmp_path: Path, -) -> None: - engine = create_async_engine(artifact_database_env) - factory = async_sessionmaker(engine, expire_on_commit=False) - first_store = _NamespaceObservingAcknowledgingStore(factory) - second_store = _NamespaceObservingAcknowledgingStore(factory) - first = artifact_storage_namespace_spec(_settings(tmp_path), first_store) - second_descriptor = dict(first.namespace_descriptor) - second_descriptor["private_root_identity"] = "sha256:" + "a" * 64 - second = ArtifactStorageNamespaceSpec( - backend=first.backend, - adapter=first.adapter, - provider_profile=first.provider_profile, - namespace_descriptor=second_descriptor, - namespace_fingerprint=canonical_json_hash(second_descriptor), - ) - - async def publish( - item_id: str, - source: CommittedArtifactSource, - store: _NamespaceObservingAcknowledgingStore, - spec: ArtifactStorageNamespaceSpec, - ) -> object: - async with factory() as session: - try: - return await ArtifactStorageOrchestrator(session, store, spec).put_reserved_item( - item_id, source - ) - except ArtifactStorageNamespaceError as exc: - return exc - - try: - async with _minted_source(tmp_path / "scratch-first", b"first") as first_source: - async with _minted_source(tmp_path / "scratch-second", b"second") as second_source: - async with factory() as session: - first_item_id = await _seed_reserved_item(session, first_source.commitment) - second_item_id = await _seed_reserved_item(session, second_source.commitment) - outcomes = await asyncio.gather( - publish(first_item_id, first_source, first_store, first), - publish(second_item_id, second_source, second_store, second), - ) - - assert sum(isinstance(value, ArtifactPutResult) for value in outcomes) == 1 - assert sum(isinstance(value, ArtifactStorageNamespaceError) for value in outcomes) == 1 - winning_index = next( - index for index, value in enumerate(outcomes) if isinstance(value, ArtifactPutResult) - ) - stores = (first_store, second_store) - specs = (first, second) - assert stores[winning_index].put_calls == 1 - assert ( - stores[winning_index].observed_namespace_fingerprint - == specs[winning_index].namespace_fingerprint + class _DriftedBootstrap: + identity = ExternalServiceAdapterIdentity( + "artifact_store", + "s3_compatible", ) - assert stores[1 - winning_index].put_calls == 0 - assert stores[1 - winning_index].observed_namespace_fingerprint is None - async with factory() as session: - namespace = await session.scalar(select(ArtifactStorageNamespace)) - assert namespace is not None - assert namespace.namespace_fingerprint == specs[winning_index].namespace_fingerprint - finally: - await engine.dispose() - - -def test_namespace_descriptor_is_canonical_and_does_not_store_local_path( - tmp_path: Path, -) -> None: - settings = _settings(tmp_path) - store = _UnavailableStore() - spec = artifact_storage_namespace_spec(settings, store) - serialized = repr(spec.namespace_descriptor) - - assert spec.backend == "local" - assert spec.provider_profile == "local-v2" - assert spec.namespace_fingerprint == canonical_json_hash(spec.namespace_descriptor) - assert str(settings.artifact_local_root) not in serialized - assert spec.namespace_descriptor["private_root_identity"].startswith("sha256:") - - -def test_namespace_descriptor_changes_when_local_root_is_replaced(tmp_path: Path) -> None: - settings = _settings(tmp_path) - first_store = LocalStorageBootstrap(LocalStorageAdapter(root=settings.artifact_local_root)) - original = artifact_storage_namespace_spec(settings, first_store) - first_store.close() - - root = settings.artifact_local_root - assert root is not None - root.rename(tmp_path / "replaced-store") - root.mkdir(mode=0o700) - - replacement_store = LocalStorageBootstrap(LocalStorageAdapter(root=root)) - replacement = artifact_storage_namespace_spec(settings, replacement_store) - replacement_store.close() - assert replacement.namespace_descriptor != original.namespace_descriptor - assert replacement.namespace_fingerprint != original.namespace_fingerprint - - -def test_store_bootstrap_requires_preprovisioned_private_root(tmp_path: Path) -> None: - with pytest.raises(ArtifactConfigurationError, match="storage is unavailable"): - LocalStorageAdapter(root=tmp_path / "missing-store") - - -def test_namespace_spec_rejects_adapter_drift(tmp_path: Path) -> None: - local_store = _UnavailableStore() - with pytest.raises(ArtifactStorageNamespaceError, match="identity"): - artifact_storage_namespace_spec(Settings(), local_store) - - -@pytest.mark.asyncio -@pytest.mark.parametrize( - "provider_failure", - [asyncio.CancelledError("cancelled"), RuntimeError("provider crashed")], -) -async def test_unexpected_or_cancelled_provider_failure_requires_replay( - tmp_path: Path, - provider_failure: BaseException, -) -> None: - store = _UnavailableStore() - store.put = AsyncMock(side_effect=provider_failure) # type: ignore[method-assign] - orchestrator = ArtifactStorageOrchestrator( - None, # type: ignore[arg-type] - store, - artifact_storage_namespace_spec(_settings(tmp_path), store), - ) - orchestrator._start_put = AsyncMock( # type: ignore[method-assign] - return_value=ProviderAttemptFence(1) - ) - orchestrator._mark_replay_required = AsyncMock() # type: ignore[method-assign] - - async with _minted_source(tmp_path / "scratch", b"provider") as source: - with pytest.raises(type(provider_failure)): - await orchestrator.put_reserved_item("item", source) - orchestrator._mark_replay_required.assert_awaited_once() # type: ignore[attr-defined] - - -@pytest.mark.asyncio -@pytest.mark.parametrize( - "finalization_failure", - [ - ArtifactInputMismatchError("mismatch"), - asyncio.CancelledError("cancelled"), - RuntimeError("database unavailable"), - ], -) -async def test_finalization_failure_never_leaves_uploading_unresolved( - tmp_path: Path, - finalization_failure: BaseException, -) -> None: - store = _UnavailableStore() - store.put = AsyncMock( # type: ignore[method-assign] - return_value=ArtifactPutResult("sha256/00/" + "0" * 62, replayed=False) - ) - orchestrator = ArtifactStorageOrchestrator( - None, # type: ignore[arg-type] - store, - artifact_storage_namespace_spec(_settings(tmp_path), store), - ) - orchestrator._start_put = AsyncMock( # type: ignore[method-assign] - return_value=ProviderAttemptFence(1) - ) - orchestrator._finalize_put = AsyncMock( # type: ignore[method-assign] - side_effect=finalization_failure - ) - orchestrator._fail_put = AsyncMock() # type: ignore[method-assign] - orchestrator._mark_replay_required = AsyncMock() # type: ignore[method-assign] - - async with _minted_source(tmp_path / "scratch", b"finalize") as source: - with pytest.raises(type(finalization_failure)): - await orchestrator.put_reserved_item("item", source) - if isinstance(finalization_failure, ArtifactInputMismatchError): - orchestrator._fail_put.assert_awaited_once() # type: ignore[attr-defined] - orchestrator._mark_replay_required.assert_not_awaited() # type: ignore[attr-defined] - else: - orchestrator._mark_replay_required.assert_awaited_once() # type: ignore[attr-defined] - orchestrator._fail_put.assert_not_awaited() # type: ignore[attr-defined] - - -@pytest.mark.asyncio -async def test_startup_namespace_validation_uses_canonical_session_factory( - tmp_path: Path, - monkeypatch: pytest.MonkeyPatch, -) -> None: - settings = _settings(tmp_path) - store = _UnavailableStore() - - class _TransactionContext: - async def __aenter__(self) -> None: - return None - - async def __aexit__(self, *_args: object) -> None: - return None - - class _Session: - def begin(self) -> _TransactionContext: - return _TransactionContext() - - session = _Session() - - class _SessionContext: - async def __aenter__(self) -> object: - return session - - async def __aexit__(self, *_args: object) -> None: - return None - - repository = object() - ensured = AsyncMock() - - def create_repository(candidate_session: object) -> object: - assert candidate_session is session - return repository - - monkeypatch.setattr( - artifact_service_module, - "get_session_factory", - lambda: lambda: _SessionContext(), - ) - monkeypatch.setattr(artifact_service_module, "ArtifactRepository", create_repository) - monkeypatch.setattr( - artifact_service_module, - "_claim_and_validate_storage_namespace", - ensured, - ) - claim = await validate_artifact_storage_namespace_at_startup(store, settings) - assert isinstance(ensured.await_args.args[1], ArtifactStorageNamespaceSpec) - assert ensured.await_args.args[0] is repository - assert claim.namespace_identity == store.namespace_identity + with pytest.raises( + ArtifactStorageNamespaceError, + match="does not match configuration", + ): + artifact_storage_namespace_spec(settings, _DriftedBootstrap()) # type: ignore[arg-type] -def test_models_remove_v1_provider_retention_and_receipt_fields() -> None: - assert not hasattr(ArtifactReplica, "retention_state") - assert not hasattr(ArtifactReplica, "provider_artifact_id") - assert not hasattr(ArtifactReplica, "provider_manifest_id") - assert not hasattr(ArtifactOperationReceipt, "provider_receipt_id") - assert not hasattr(ArtifactOperationReceipt, "retention_reference") - assert not hasattr(ArtifactOperationReceipt, "service_principal") - assert not hasattr(ArtifactUploadItem, "provider_operation_reference") - assert hasattr(ArtifactReplica, "provider_object_ref") - assert hasattr(ArtifactStorageNamespace, "namespace_fingerprint") +def test_models_retain_only_v2_provider_evidence_fields() -> None: + assert "provider_manifest_id" not in ArtifactReplica.__table__.columns + assert "retention_state" not in ArtifactReplica.__table__.columns + assert "provider_receipt_id" not in ArtifactOperationReceipt.__table__.columns + assert "retention_reference" not in ArtifactOperationReceipt.__table__.columns + assert "namespace_fingerprint" in ArtifactStorageNamespace.__table__.columns diff --git a/backend/tests/test_aws_credential_isolation.py b/backend/tests/test_aws_credential_isolation.py index 40449d6c8..89da07a32 100644 --- a/backend/tests/test_aws_credential_isolation.py +++ b/backend/tests/test_aws_credential_isolation.py @@ -13,12 +13,14 @@ from app.adapters.artifacts import s3_compatible from app.core.config import Settings +from tests.artifact_store_helpers import artifact_admission_limit_settings from app.interfaces.artifacts import ArtifactConfigurationError from tests.assertion_helpers import assert_secret_not_retained def _aws_settings(tmp_path: Path, method: str = "container-role") -> Settings: return Settings( + **artifact_admission_limit_settings(), environment="production", artifact_store_backend="s3_compatible", artifact_scratch_root=tmp_path / "scratch", diff --git a/backend/tests/test_config.py b/backend/tests/test_config.py index 8aec41452..04e65ce97 100644 --- a/backend/tests/test_config.py +++ b/backend/tests/test_config.py @@ -34,6 +34,7 @@ ) from app.main import create_app from tests.assertion_helpers import assert_secret_not_retained +from tests.artifact_store_helpers import artifact_admission_limit_settings def test_default_settings_are_fail_closed(monkeypatch: pytest.MonkeyPatch) -> None: @@ -386,6 +387,7 @@ def _flow_settings(**overrides) -> Settings: def _minio_setting_values(tmp_path: Path, **overrides: object) -> dict[str, object]: values: dict[str, object] = { + **artifact_admission_limit_settings(), "environment": "test", "artifact_store_backend": "s3_compatible", "artifact_scratch_root": tmp_path / "scratch", @@ -409,6 +411,7 @@ def _minio_settings(tmp_path: Path, **overrides: object) -> Settings: def _aws_settings(tmp_path: Path, **overrides: object) -> Settings: values: dict[str, object] = { + **artifact_admission_limit_settings(), "environment": "production", "artifact_store_backend": "s3_compatible", "artifact_scratch_root": tmp_path / "scratch", @@ -529,6 +532,7 @@ def test_settings_reject_local_artifacts_outside_development(environment: str) - """Keep filesystem storage out of production-like deployments.""" with pytest.raises(ValidationError, match="local artifact storage"): Settings( + **artifact_admission_limit_settings(), environment=environment, artifact_store_backend="local", artifact_local_root="/tmp/workstream-artifacts", @@ -539,11 +543,38 @@ def test_settings_reject_local_artifacts_outside_development(environment: str) - def test_settings_require_scratch_root_for_enabled_artifacts() -> None: with pytest.raises(ValidationError, match="artifact scratch root"): Settings( + **artifact_admission_limit_settings(), environment="test", artifact_store_backend="s3_compatible", ) +def test_settings_require_every_durable_byte_admission_limit(tmp_path: Path) -> None: + """Enabled stores must never inherit an implicit capacity policy.""" + common = { + "environment": "test", + "artifact_store_backend": "local", + "artifact_local_root": tmp_path / "artifacts", + "artifact_scratch_root": tmp_path / "scratch", + } + with pytest.raises( + ValidationError, + match="requires all durable-byte admission limits", + ): + Settings(**common) + + partial = artifact_admission_limit_settings() + partial.pop("artifact_admission_task_maximum_bytes") + with pytest.raises( + ValidationError, + match="requires all durable-byte admission limits", + ): + Settings(**common, **partial) + + settings = Settings(**common, **artifact_admission_limit_settings()) + assert settings.artifact_admission_task_maximum_bytes == 1024 + + @pytest.mark.parametrize("interval", [0, -1, 86_401]) def test_artifact_scratch_cleanup_interval_is_positive_and_bounded(interval: int) -> None: with pytest.raises(ValidationError): @@ -565,6 +596,7 @@ def test_artifact_backend_enum_is_exact_and_flow_node_is_rejected(tmp_path: Path def test_local_artifact_settings_and_factory(tmp_path: Path) -> None: """Construct local storage only from complete development configuration.""" settings = Settings( + **artifact_admission_limit_settings(), environment="test", artifact_store_backend="local", artifact_local_root=tmp_path / "artifacts", @@ -992,6 +1024,7 @@ def test_minio_secret_values_from_env_and_dotenv_are_absent_from_errors( with pytest.raises(ValidationError) as env_error: Settings( + **artifact_admission_limit_settings(), environment="test", artifact_store_backend="s3_compatible", artifact_scratch_root=tmp_path / "scratch", @@ -1009,6 +1042,7 @@ def test_minio_secret_values_from_env_and_dotenv_are_absent_from_errors( monkeypatch.delenv("WORKSTREAM_ARTIFACT_S3_SECRET_ACCESS_KEY") with pytest.raises(ValidationError) as dotenv_error: Settings( + **artifact_admission_limit_settings(), environment="test", artifact_store_backend="s3_compatible", artifact_scratch_root=tmp_path / "scratch", @@ -1026,6 +1060,7 @@ def test_minio_secret_values_from_env_and_dotenv_are_absent_from_errors( def test_minio_endpoint_is_normalized_before_namespace_identity(tmp_path: Path) -> None: """Equivalent MinIO origins must produce one endpoint and namespace identity.""" common = { + **artifact_admission_limit_settings(), "environment": "test", "artifact_store_backend": "s3_compatible", "artifact_scratch_root": tmp_path / "scratch", diff --git a/backend/tests/test_s3_artifact_store.py b/backend/tests/test_s3_artifact_store.py index 054f601c7..336ccb653 100644 --- a/backend/tests/test_s3_artifact_store.py +++ b/backend/tests/test_s3_artifact_store.py @@ -37,7 +37,10 @@ from app.modules.artifacts.preparation import ArtifactPreparationService from app.modules.artifacts.sources import ArtifactCommitment from tests.assertion_helpers import assert_secret_not_retained -from tests.artifact_store_helpers import minted_source +from tests.artifact_store_helpers import ( + artifact_admission_limit_settings, + minted_source, +) from tests.test_artifact_store_conformance import ArtifactStoreConformanceTests @@ -91,6 +94,7 @@ def minio_settings( ) -> Settings: """Return one complete local-only MinIO configuration.""" return Settings( + **artifact_admission_limit_settings(), environment="test", artifact_store_backend="s3_compatible", artifact_scratch_root=Path("/tmp/workstream-test-artifact-scratch"), diff --git a/docs/spec_artifact_storage_service.md b/docs/spec_artifact_storage_service.md index 13f9d3c0a..615725fe3 100644 --- a/docs/spec_artifact_storage_service.md +++ b/docs/spec_artifact_storage_service.md @@ -724,10 +724,11 @@ in the v2 clean cut. No compatibility adapter or dual format remains. provider call. 3. Transaction A atomically claims or validates the deployment storage namespace, derives and reserves every applicable durable-storage scope - charge as `provisional`, reserves the item, and commits the server-computed - digest, size, media type, operation identity, canonical request digest, - limits, one `ArtifactPutAttempt`, and CAS. Any quota or namespace failure - rolls back all reservations. + charge as `provisional`, links the authoritative producer source reference, + and commits the server-computed digest, size, media type, operation identity, + canonical request digest, exact limits, one `ArtifactPutAttempt`, and CAS. + Any quota, relationship, or namespace failure rolls back the complete + transaction. 4. Workstream passes the sealed `CommittedArtifactSource` from the artifact orchestration service to the adapter outside the transaction. @@ -746,6 +747,12 @@ in the v2 clean cut. No compatibility adapter or dual format remains. No provider call occurs inside a PostgreSQL transaction. No product binding is created before independent verification. +Chunk 02C1 implements step 3 only. It creates a `prepared` attempt with no +executor, lease, receipt, replica, or execution generation and exposes no +product route. Steps 4 through 9 remain inactive until their separately +approved owning chunks. In particular, 02C1 contains no provider `put`, +`observe_put_result`, verification publication, recovery, or product cutover. + Acknowledgement loss leaves the durable put attempt and charges provisional. The attempt scanner publishes resolution after an ambiguous outcome or expired execution lease. Resolution performs a fresh read-only observation and full @@ -808,6 +815,20 @@ The preparation settings use the standard `WORKSTREAM_` environment prefix: | `WORKSTREAM_ARTIFACT_STREAM_BUFFER_BYTES` | `1048576` | Bounded streaming buffer, limited to at most 1 MiB. | | `WORKSTREAM_ARTIFACT_OPERATION_LOCK_TIMEOUT_SECONDS` | `1800` | Maximum wait for a private cross-process artifact-store operation lock before failing closed. | +Enabled artifact storage also requires an explicit durable-byte policy. None +of these limits has a default, and startup fails unless all four are positive: + +| Environment variable | Default | Contract | +|---|---:|---| +| `WORKSTREAM_ARTIFACT_ADMISSION_TASK_MAXIMUM_BYTES` | unset | Maximum cumulative unique provisional/completed bytes charged to one task. Guide bytes have no task scope. | +| `WORKSTREAM_ARTIFACT_ADMISSION_PRODUCER_MAXIMUM_BYTES` | unset | Maximum cumulative unique provisional/completed bytes charged to one canonical human actor or fixed service identity. | +| `WORKSTREAM_ARTIFACT_ADMISSION_PROJECT_MAXIMUM_BYTES` | unset | Maximum cumulative unique provisional/completed bytes charged to one project. | +| `WORKSTREAM_ARTIFACT_ADMISSION_DEPLOYMENT_MAXIMUM_BYTES` | unset | Maximum cumulative unique provisional/completed bytes charged to the deployment namespace. | + +The first committed scope row pins its configured limit. A later process whose +configuration disagrees fails admission instead of silently changing the +persisted capacity contract. + The S3-compatible provider settings use the same prefix: | Environment variable | Default | Contract | diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 60eb26719..c631c5744 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -4154,7 +4154,7 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: ).read_text(encoding="utf-8") artifact_contract = Path( ".agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/" - "WS-ART-001-02B1-s3-compatible-minio-aws.md" + "WS-ART-001-02C1-admission-put-attempt-foundation.md" ).read_text(encoding="utf-8") work_queue = Path(".agent-loop/WORK_QUEUE.md").read_text(encoding="utf-8") loop_state = Path(".agent-loop/LOOP_STATE.md").read_text(encoding="utf-8") @@ -4223,14 +4223,15 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: ) assert "Merged through PR #129 as `9a04434`" in artifact_map assert "Merged through PR #141 as `a10d901`" in artifact_map - assert "Active after 02A3 merged through PR #141" in artifact_map - assert "Status: Active after explicit human start" in artifact_contract + assert "Merged through PR #151 as `1b5422fc`" in artifact_map + assert "Active after PR #151 and explicit user start" in artifact_map + assert "Status: Active after explicit user start" in artifact_contract assert ( "AUTH's owner reconciliation merged through PR #140 as\n" "`d541521`" in artifact_status ) - assert "`WS-ART-001-02B1` is active" in artifact_status - assert "The current gate is deterministic 02B1 proof followed by all nine" in ( + assert "`WS-ART-001-02C1` is active" in artifact_status + assert "The current gate is deterministic 02C1 proof followed by all nine" in ( artifact_status.replace("\n", " ") ) assert "No later artifact chunk starts automatically" in artifact_status.replace( @@ -4240,22 +4241,12 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: "| `WS-AUTH-001-09C` | Actor And Identity-Link Administration Reads | L1 | " "Merged through PR #146 as `0ffdabf`" in work_queue ) - assert ( - "| `WS-ART-001-02B1` | S3-Compatible MinIO And AWS | L1 | " - "Merged through PR #151 as `1b5422f` on 2026-07-19" in work_queue - ) - assert ( - "PR #151 then merged `WS-ART-001-02B1` as `1b5422f` on 2026-07-19" - in loop_state - ) - assert ( - "ART-02C1 remains inactive pending signed memory and a separate explicit start" - in loop_state.replace("\n", " ") - ) - assert "Current ART gate: integrate trusted `main`" not in loop_state - assert "| `WS-ART-001-02B1` | S3-Compatible MinIO And AWS | L1 | Active" not in ( + assert "| `WS-ART-001-02C1` | Admission And Put-Attempt Foundation | L1 | Active" in ( work_queue ) + assert "Current ART gate: integrate trusted `main`, complete deterministic 02C1" in ( + loop_state + ) def test_stale_authorization_discovery_includes_new_untracked_docs() -> None: From 9596637d9bfeac53f3eff863b0f6a91303a08bee Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 04:51:38 +0100 Subject: [PATCH 02/20] Bind guide admission to snapshot capturer --- backend/app/modules/artifacts/repository.py | 3 +++ backend/app/modules/artifacts/service.py | 3 ++- backend/tests/test_artifact_admission.py | 24 ++++++++++++++++++++- 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/backend/app/modules/artifacts/repository.py b/backend/app/modules/artifacts/repository.py index 0b881d662..010fcc3fb 100644 --- a/backend/app/modules/artifacts/repository.py +++ b/backend/app/modules/artifacts/repository.py @@ -34,6 +34,7 @@ class GuideAdmissionFacts: guide_source_item_id: str project_id: str + captured_by: str content_hash: str media_type: str @@ -115,6 +116,7 @@ async def get_guide_admission_facts( select( GuideSourceSnapshotItem.id, GuideSourceSnapshot.project_id, + GuideSourceSnapshot.captured_by, GuideSourceSnapshotItem.content_hash, GuideSourceSnapshotItem.media_type, ) @@ -130,6 +132,7 @@ async def get_guide_admission_facts( return GuideAdmissionFacts( guide_source_item_id=row.id, project_id=row.project_id, + captured_by=row.captured_by, content_hash=row.content_hash, media_type=row.media_type, ) diff --git a/backend/app/modules/artifacts/service.py b/backend/app/modules/artifacts/service.py index 9938c2c64..c5376ac17 100644 --- a/backend/app/modules/artifacts/service.py +++ b/backend/app/modules/artifacts/service.py @@ -326,6 +326,7 @@ async def _guide_facts( commitment = request.source.commitment if ( row is None + or row.captured_by != str(context.actor_profile_id) or row.content_hash != commitment.sha256 or row.media_type != commitment.media_type ): @@ -338,7 +339,7 @@ async def _guide_facts( return _AdmissionFacts( request_type="guide", producer_type="actor_profile", - producer_ref=str(context.actor_profile_id), + producer_ref=row.captured_by, project_id=row.project_id, task_id=None, guide_source_item_id=item_id, diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index 149b13086..f6fd8c5a9 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -141,6 +141,7 @@ def _context( async def _seed_guide( session, *, + captured_by: str, content_hash: str, media_type: str, ) -> tuple[str, str]: @@ -176,7 +177,7 @@ async def _seed_guide( manifest_schema_version="v1", manifest_json={"items": [item_id]}, bundle_hash=canonical_json_hash({"items": [item_id]}), - captured_by="test", + captured_by=captured_by, ) ) await session.flush() @@ -298,9 +299,30 @@ async def test_guide_admission_derives_three_scopes_without_provider_evidence( ) as source: project_id, item_id = await _seed_guide( session, + captured_by=str(context.actor_profile_id), content_hash=source.commitment.sha256, media_type=source.commitment.media_type, ) + with pytest.raises( + ArtifactAdmissionRelationshipError, + match="guide source item relationship is unavailable", + ): + await ArtifactAdmissionService( + session, + settings, + namespace, + ).admit( + GuideArtifactAdmissionRequest( + authorization_context=_context(), + guide_source_item_id=UUID(item_id), + source=source, + ) + ) + assert await _count(session, ArtifactStorageNamespace) == 0 + assert await _count(session, ArtifactAdmissionScope) == 0 + assert await _count(session, ArtifactAdmissionCharge) == 0 + assert await _count(session, ArtifactPutAttempt) == 0 + await session.rollback() result = await ArtifactAdmissionService( session, settings, From b91f842fd6244b7679071d001a8bfe185659aa82 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 05:21:26 +0100 Subject: [PATCH 03/20] Prove checker output admission end to end --- backend/tests/test_artifact_admission.py | 342 +++++++++++++++++++++-- 1 file changed, 318 insertions(+), 24 deletions(-) diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index f6fd8c5a9..b05e1cdbb 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -5,7 +5,6 @@ import asyncio from datetime import UTC, datetime, timedelta from pathlib import Path -from unittest.mock import AsyncMock from uuid import UUID, uuid4 from alembic import command @@ -19,6 +18,7 @@ from app.core.hashing import canonical_json_hash from app.modules.actors.models import ActorIdentityLink, ActorProfile from app.modules.actors.service_identities import ServiceIdentity +from app.modules.checkers.models import CheckerRun from app.modules.artifacts.models import ( ArtifactAdmissionCharge, ArtifactAdmissionScope, @@ -31,7 +31,6 @@ ArtifactUploadItem, ArtifactUploadSession, ) -from app.modules.artifacts.repository import CheckerOutputAdmissionFacts from app.modules.artifacts.schemas import ( CheckerOutputArtifactAdmissionRequest, ContributorArtifactAdmissionRequest, @@ -52,12 +51,19 @@ IdentityLinkStatus, ) from app.modules.projects.models import ( + EffectiveProjectSubmissionArtifactPolicy, GuideSourceSnapshot, GuideSourceSnapshotItem, + PaymentPolicy, + PostSubmitCheckerPolicy, + PreSubmitCheckerPolicy, Project, ProjectGuide, + ReviewPolicy, + RevisionPolicy, + SubmissionArtifactPolicy, ) -from app.modules.tasks.models import WorkstreamTask +from app.modules.tasks.models import Submission, WorkstreamTask from tests.artifact_store_helpers import ( artifact_admission_limit_settings, minted_source, @@ -275,6 +281,252 @@ async def _seed_contributor_items( return project_id, task_id, tuple(item_ids) +async def _seed_checker_output_relationships(session) -> tuple[str, str, str]: + """Persist one complete checker-run ownership chain for admission proof.""" + project_id = str(uuid4()) + guide_id = str(uuid4()) + snapshot_id = str(uuid4()) + submission_policy_id = str(uuid4()) + effective_policy_id = str(uuid4()) + pre_submit_policy_id = str(uuid4()) + post_submit_policy_id = str(uuid4()) + task_id = str(uuid4()) + submission_id = str(uuid4()) + checker_run_id = str(uuid4()) + guide_version = "v1" + snapshot_hash = canonical_json_hash({"items": []}) + submission_policy_body = {"required_artifacts": []} + submission_policy_hash = canonical_json_hash(submission_policy_body) + effective_policy_body = {"required_artifacts": [], "artifact_hash_algorithm": "sha256"} + effective_policy_hash = canonical_json_hash(effective_policy_body) + pre_submit_bundle = {"schema_version": "v1", "rules": []} + pre_submit_bundle_hash = canonical_json_hash(pre_submit_bundle) + post_submit_policy_body = {"required_checkers": []} + post_submit_policy_hash = canonical_json_hash(post_submit_policy_body) + now = datetime.now(UTC) + + session.add(Project(id=project_id, name="Checker project", slug=f"checker-{project_id}")) + await session.flush() + session.add( + ProjectGuide( + id=guide_id, + project_id=project_id, + version=guide_version, + status="active", + content_markdown="# Checker guide", + approved_by="setup-actor", + effective_at=now, + created_by="setup-actor", + ) + ) + await session.flush() + session.add( + GuideSourceSnapshot( + id=snapshot_id, + project_id=project_id, + guide_id=guide_id, + guide_version=guide_version, + manifest_schema_version="v1", + manifest_json={"items": []}, + bundle_hash=snapshot_hash, + captured_by="setup-actor", + ) + ) + await session.flush() + session.add( + SubmissionArtifactPolicy( + id=submission_policy_id, + project_id=project_id, + guide_id=guide_id, + guide_version=guide_version, + source_snapshot_id=snapshot_id, + source_snapshot_hash=snapshot_hash, + policy_version="v1", + lifecycle_status="approved", + policy_body=submission_policy_body, + policy_hash=submission_policy_hash, + derivation_source="test", + source_material_refs=[], + created_by="setup-actor", + approved_by_role="admin", + approved_by_actor="setup-actor", + approved_at=now, + ) + ) + await session.flush() + session.add( + EffectiveProjectSubmissionArtifactPolicy( + id=effective_policy_id, + project_id=project_id, + guide_id=guide_id, + guide_version=guide_version, + source_snapshot_id=snapshot_id, + source_snapshot_hash=snapshot_hash, + submission_artifact_policy_id=submission_policy_id, + submission_artifact_policy_hash=submission_policy_hash, + lifecycle_status="approved", + merge_algorithm_version="v1", + effective_policy=effective_policy_body, + effective_policy_hash=effective_policy_hash, + created_by="setup-actor", + ) + ) + await session.flush() + session.add( + PreSubmitCheckerPolicy( + id=pre_submit_policy_id, + project_id=project_id, + guide_id=guide_id, + guide_version=guide_version, + source_snapshot_id=snapshot_id, + source_snapshot_hash=snapshot_hash, + effective_policy_id=effective_policy_id, + effective_policy_hash=effective_policy_hash, + lifecycle_status="compiled", + compiler_version="v1", + compiled_bundle=pre_submit_bundle, + compiled_bundle_hash=pre_submit_bundle_hash, + checker_names=[], + checker_configs={}, + created_by="setup-actor", + ) + ) + await session.flush() + session.add_all( + [ + PostSubmitCheckerPolicy( + id=post_submit_policy_id, + project_id=project_id, + guide_id=guide_id, + guide_version=guide_version, + source_snapshot_id=snapshot_id, + source_snapshot_hash=snapshot_hash, + effective_policy_id=effective_policy_id, + effective_policy_hash=effective_policy_hash, + pre_submit_checker_policy_id=pre_submit_policy_id, + pre_submit_checker_bundle_hash=pre_submit_bundle_hash, + required_checkers=[], + warning_checkers=[], + blocking_severities=["error"], + policy_hash=post_submit_policy_hash, + policy_body=post_submit_policy_body, + lifecycle_status="approved", + approved_by_role="admin", + approved_by_actor="setup-actor", + approved_at=now, + created_by="setup-actor", + ), + ReviewPolicy( + id=str(uuid4()), + project_id=project_id, + guide_version=guide_version, + requires_second_review=False, + allowed_decisions=["accept", "needs_revision", "reject"], + minimum_finding_fields=[], + ), + RevisionPolicy( + id=str(uuid4()), + project_id=project_id, + guide_version=guide_version, + max_revision_rounds=1, + revision_deadline_hours=24, + auto_reject_after_limit=True, + allowed_resubmission_states=["needs_revision"], + ), + PaymentPolicy( + id=str(uuid4()), + project_id=project_id, + guide_version=guide_version, + ), + ] + ) + await session.flush() + session.add( + WorkstreamTask( + id=task_id, + project_id=project_id, + locked_guide_version=guide_version, + locked_post_submit_checker_policy_id=post_submit_policy_id, + locked_post_submit_checker_policy_version=guide_version, + locked_post_submit_checker_policy_hash=post_submit_policy_hash, + locked_post_submit_checker_policy_body=post_submit_policy_body, + locked_review_policy_version=guide_version, + locked_revision_policy_version=guide_version, + locked_payment_policy_version=guide_version, + locked_guide_source_snapshot_id=snapshot_id, + locked_guide_source_snapshot_hash=snapshot_hash, + locked_effective_project_submission_artifact_policy_id=effective_policy_id, + locked_effective_project_submission_artifact_policy_hash=effective_policy_hash, + locked_pre_submit_checker_policy_id=pre_submit_policy_id, + locked_pre_submit_checker_bundle_hash=pre_submit_bundle_hash, + title="Checker admission task", + description="Prove checker output admission.", + status="draft", + created_by="setup-actor", + ) + ) + await session.flush() + session.add( + Submission( + id=submission_id, + task_id=task_id, + worker_id=str(uuid4()), + version=1, + status="submitted", + summary="Checker source submission", + package_hash=canonical_json_hash({"submission": submission_id}), + artifact_hash_manifest=[], + worker_attestation="complete", + locked_guide_version=guide_version, + locked_post_submit_checker_policy_id=post_submit_policy_id, + locked_post_submit_checker_policy_version=guide_version, + locked_post_submit_checker_policy_hash=post_submit_policy_hash, + locked_post_submit_checker_policy_body=post_submit_policy_body, + locked_review_policy_version=guide_version, + locked_revision_policy_version=guide_version, + locked_payment_policy_version=guide_version, + locked_guide_source_snapshot_id=snapshot_id, + locked_guide_source_snapshot_hash=snapshot_hash, + locked_effective_project_submission_artifact_policy_id=effective_policy_id, + locked_effective_project_submission_artifact_policy_hash=effective_policy_hash, + locked_pre_submit_checker_policy_id=pre_submit_policy_id, + locked_pre_submit_checker_bundle_hash=pre_submit_bundle_hash, + ) + ) + await session.flush() + session.add( + CheckerRun( + id=checker_run_id, + task_id=task_id, + submission_id=submission_id, + submission_version=1, + trigger_source="submission_finalized", + status="queued", + routing_recommendation="not_evaluated", + outcome_source="none", + triggered_by="setup-actor", + triggered_by_subject="setup-subject", + triggered_by_issuer="https://issuer.example.test", + trigger_auth_source="test", + attempt_number=1, + is_current_for_submission=True, + locked_guide_version=guide_version, + locked_post_submit_checker_policy_id=post_submit_policy_id, + locked_post_submit_checker_policy_version=guide_version, + locked_post_submit_checker_policy_hash=post_submit_policy_hash, + locked_post_submit_checker_policy_body=post_submit_policy_body, + locked_review_policy_version=guide_version, + locked_revision_policy_version=guide_version, + locked_payment_policy_version=guide_version, + package_hash=canonical_json_hash({"submission": submission_id}), + artifact_hash_manifest=[], + artifact_manifest_hash=canonical_json_hash([]), + ) + ) + await session.commit() + return project_id, task_id, checker_run_id + + async def _count(session, model: type) -> int: value = await session.scalar(select(func.count()).select_from(model)) assert value is not None @@ -748,6 +1000,9 @@ async def test_checker_output_requires_exact_active_fixed_service_identity( factory = async_sessionmaker(engine, expire_on_commit=False) try: async with factory() as session: + project_id, task_id, checker_run_id = await _seed_checker_output_relationships( + session + ) session.add( ActorProfile( id=str(actor_id), @@ -772,38 +1027,77 @@ async def test_checker_output_requires_exact_active_fixed_service_identity( await session.commit() async with minted_source(tmp_path / "scratch-source", b"checker") as source: service = ArtifactAdmissionService(session, settings, namespace) - service._repo.get_checker_output_admission_facts = AsyncMock( - return_value=CheckerOutputAdmissionFacts( - checker_run_id="run-id", - project_id="project-id", - task_id="task-id", - ) - ) - facts = await service._checker_output_facts( - CheckerOutputArtifactAdmissionRequest( - authorization_context=context, - checker_run_id=uuid4(), - logical_role="platform-review", - source=source, - ) - ) - assert facts.producer_ref == ( - ServiceIdentity.ARTIFACT_CHECKER_OUTPUT.value - ) - forged = context.model_copy(update={"identity_link_id": uuid4()}) with pytest.raises( ArtifactAdmissionRelationshipError, match="service identity is unavailable", ): - await service._checker_output_facts( + await service.admit( CheckerOutputArtifactAdmissionRequest( authorization_context=forged, - checker_run_id=uuid4(), + checker_run_id=UUID(checker_run_id), logical_role="platform-review", source=source, ) ) + assert await _count(session, ArtifactStorageNamespace) == 0 + assert await _count(session, ArtifactAdmissionScope) == 0 + assert await _count(session, ArtifactAdmissionCharge) == 0 + assert await _count(session, ArtifactPutAttempt) == 0 + await session.rollback() + + request = CheckerOutputArtifactAdmissionRequest( + authorization_context=context, + checker_run_id=UUID(checker_run_id), + logical_role="platform-review", + source=source, + ) + result = await service.admit(request) + replay = await service.admit(request) + + attempt = await session.get(ArtifactPutAttempt, str(result.attempt_id)) + scopes = ( + await session.execute( + select(ArtifactAdmissionScope).order_by( + ArtifactAdmissionScope.scope_type, + ArtifactAdmissionScope.scope_id, + ) + ) + ).scalars().all() + links = ( + await session.execute( + select(ArtifactPutAttemptCharge).where( + ArtifactPutAttemptCharge.attempt_id == str(result.attempt_id) + ) + ) + ).scalars().all() + assert attempt is not None + assert replay.attempt_id == result.attempt_id + assert replay.charge_ids == result.charge_ids + assert attempt.status == "prepared" + assert attempt.producer_request_type == "checker_output" + assert attempt.producer_type == "service_identity" + assert attempt.producer_ref == ServiceIdentity.ARTIFACT_CHECKER_OUTPUT.value + assert attempt.project_id == project_id + assert attempt.task_id == task_id + assert attempt.checker_run_id == checker_run_id + assert attempt.logical_role == "platform-review" + assert attempt.executor_id is None + assert attempt.lease_expires_at is None + assert attempt.execution_generation == 0 + assert {scope.scope_type for scope in scopes} == { + "deployment", + "producer", + "project", + "task", + } + assert len(result.charge_ids) == 4 + assert len(links) == 4 + assert await _count(session, ArtifactPutAttempt) == 1 + assert await _count(session, ArtifactAdmissionCharge) == 4 + assert await _count(session, ArtifactContent) == 0 + assert await _count(session, ArtifactReplica) == 0 + assert await _count(session, ArtifactOperationReceipt) == 0 finally: await engine.dispose() From 26c0aad73669bad1f2f1d8c93616328d12774941 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 05:42:16 +0100 Subject: [PATCH 04/20] Close artifact admission relationship gaps --- backend/app/modules/artifacts/repository.py | 13 +++- backend/app/modules/artifacts/service.py | 1 + backend/tests/test_artifact_admission.py | 77 +++++++++++++++++++++ 3 files changed, 88 insertions(+), 3 deletions(-) diff --git a/backend/app/modules/artifacts/repository.py b/backend/app/modules/artifacts/repository.py index 010fcc3fb..c80161053 100644 --- a/backend/app/modules/artifacts/repository.py +++ b/backend/app/modules/artifacts/repository.py @@ -145,8 +145,8 @@ async def get_contributor_admission_facts( await self._session.execute( select( ArtifactUploadItem.id, - ArtifactUploadSession.project_id, - ArtifactUploadSession.task_id, + WorkstreamTask.project_id, + WorkstreamTask.id.label("task_id"), ArtifactUploadSession.actor_id, ArtifactUploadSession.state.label("session_state"), ArtifactUploadItem.state.label("item_state"), @@ -158,8 +158,15 @@ async def get_contributor_admission_facts( ArtifactUploadSession, ArtifactUploadSession.id == ArtifactUploadItem.session_id, ) + .join( + WorkstreamTask, + (WorkstreamTask.id == ArtifactUploadSession.task_id) + & (WorkstreamTask.project_id == ArtifactUploadSession.project_id), + ) .where(ArtifactUploadItem.id == upload_item_id) - .with_for_update(of=(ArtifactUploadSession, ArtifactUploadItem)) + .with_for_update( + of=(ArtifactUploadSession, ArtifactUploadItem, WorkstreamTask) + ) ) ).one_or_none() if row is None: diff --git a/backend/app/modules/artifacts/service.py b/backend/app/modules/artifacts/service.py index c5376ac17..878ff462c 100644 --- a/backend/app/modules/artifacts/service.py +++ b/backend/app/modules/artifacts/service.py @@ -448,6 +448,7 @@ def _validate_logical_role(value: str) -> str: not isinstance(value, str) or value != value.strip() or not value + or not value.isascii() or len(value) > 100 or any(ord(character) < 32 or ord(character) == 127 for character in value) ): diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index b05e1cdbb..75ad99f27 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -686,6 +686,65 @@ async def test_exact_replay_returns_one_attempt_and_one_charge_set( await engine.dispose() +async def test_contributor_admission_rejects_cross_project_task_relationship( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path) + namespace = _namespace(settings) + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + async with minted_source(tmp_path / "scratch-source", b"contributor") as source: + _, _, item_ids = await _seed_contributor_items( + session, + actor_profile_id=str(context.actor_profile_id), + commitments=( + ( + source.commitment.sha256, + source.commitment.byte_count, + source.commitment.media_type, + ), + ), + ) + item = await session.get(ArtifactUploadItem, item_ids[0]) + assert item is not None + upload_session = await session.get(ArtifactUploadSession, item.session_id) + assert upload_session is not None + unrelated_project_id = str(uuid4()) + session.add( + Project( + id=unrelated_project_id, + name="Unrelated admission project", + slug=f"unrelated-{unrelated_project_id}", + ) + ) + await session.flush() + upload_session.project_id = unrelated_project_id + await session.commit() + + with pytest.raises( + ArtifactAdmissionRelationshipError, + match="contributor upload item relationship is unavailable", + ): + await ArtifactAdmissionService(session, settings, namespace).admit( + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[0]), + source=source, + ) + ) + + assert await _count(session, ArtifactStorageNamespace) == 0 + assert await _count(session, ArtifactAdmissionScope) == 0 + assert await _count(session, ArtifactAdmissionCharge) == 0 + assert await _count(session, ArtifactPutAttempt) == 0 + finally: + await engine.dispose() + + async def test_same_content_distinct_operations_deduplicate_scope_bytes( admission_database_env: str, tmp_path: Path, @@ -1027,6 +1086,24 @@ async def test_checker_output_requires_exact_active_fixed_service_identity( await session.commit() async with minted_source(tmp_path / "scratch-source", b"checker") as source: service = ArtifactAdmissionService(session, settings, namespace) + with pytest.raises( + ArtifactAdmissionRelationshipError, + match="logical role is invalid", + ): + await service.admit( + CheckerOutputArtifactAdmissionRequest( + authorization_context=context, + checker_run_id=UUID(checker_run_id), + logical_role="é" * 100, + source=source, + ) + ) + assert await _count(session, ArtifactStorageNamespace) == 0 + assert await _count(session, ArtifactAdmissionScope) == 0 + assert await _count(session, ArtifactAdmissionCharge) == 0 + assert await _count(session, ArtifactPutAttempt) == 0 + await session.rollback() + forged = context.model_copy(update={"identity_link_id": uuid4()}) with pytest.raises( ArtifactAdmissionRelationshipError, From af5a1c147bb10ad978262a0aeea29245877df29b Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 06:10:01 +0100 Subject: [PATCH 05/20] Bind checker output to canonical submission task --- backend/app/modules/artifacts/repository.py | 13 ++- backend/app/modules/artifacts/service.py | 4 +- backend/tests/test_artifact_admission.py | 116 +++++++++++++++++++- 3 files changed, 123 insertions(+), 10 deletions(-) diff --git a/backend/app/modules/artifacts/repository.py b/backend/app/modules/artifacts/repository.py index c80161053..d6f07de4b 100644 --- a/backend/app/modules/artifacts/repository.py +++ b/backend/app/modules/artifacts/repository.py @@ -25,7 +25,7 @@ ) from app.modules.checkers.models import CheckerRun from app.modules.projects.models import GuideSourceSnapshot, GuideSourceSnapshotItem -from app.modules.tasks.models import WorkstreamTask +from app.modules.tasks.models import Submission, WorkstreamTask @dataclass(frozen=True, slots=True) @@ -189,9 +189,16 @@ async def get_checker_output_admission_facts( """Load canonical project/task ownership for one checker run.""" row = ( await self._session.execute( - select(CheckerRun.id, CheckerRun.task_id, WorkstreamTask.project_id) - .join(WorkstreamTask, WorkstreamTask.id == CheckerRun.task_id) + select(CheckerRun.id, Submission.task_id, WorkstreamTask.project_id) + .join( + Submission, + (Submission.id == CheckerRun.submission_id) + & (Submission.version == CheckerRun.submission_version) + & (Submission.task_id == CheckerRun.task_id), + ) + .join(WorkstreamTask, WorkstreamTask.id == Submission.task_id) .where(CheckerRun.id == checker_run_id) + .with_for_update(of=(CheckerRun, Submission, WorkstreamTask)) ) ).one_or_none() if row is None: diff --git a/backend/app/modules/artifacts/service.py b/backend/app/modules/artifacts/service.py index 878ff462c..371060ddd 100644 --- a/backend/app/modules/artifacts/service.py +++ b/backend/app/modules/artifacts/service.py @@ -291,6 +291,8 @@ def _validate_request_boundary(request: ArtifactAdmissionRequest) -> None: raise TypeError("invalid artifact admission authorization context") if type(request.source) is not CommittedArtifactSource: raise TypeError("invalid artifact admission source") + if type(request) is CheckerOutputArtifactAdmissionRequest: + ArtifactAdmissionService._validate_logical_role(request.logical_role) context = request.authorization_context if ( context.actor_status is not ActorStatus.ACTIVE @@ -399,7 +401,7 @@ async def _checker_output_facts( raise ArtifactAdmissionRelationshipError( "checker output producer must be a service actor" ) - logical_role = self._validate_logical_role(request.logical_role) + logical_role = request.logical_role service_actor = await self._repo.lock_service_actor( str(context.actor_profile_id) ) diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index 75ad99f27..ab11b1292 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -1084,17 +1084,75 @@ async def test_checker_output_requires_exact_active_fixed_service_identity( ) ) await session.commit() + canonical_task = await session.get(WorkstreamTask, task_id) + assert canonical_task is not None + unrelated_task_id = str(uuid4()) + session.add( + WorkstreamTask( + id=unrelated_task_id, + project_id=project_id, + locked_guide_version=canonical_task.locked_guide_version, + locked_post_submit_checker_policy_id=( + canonical_task.locked_post_submit_checker_policy_id + ), + locked_post_submit_checker_policy_version=( + canonical_task.locked_post_submit_checker_policy_version + ), + locked_post_submit_checker_policy_hash=( + canonical_task.locked_post_submit_checker_policy_hash + ), + locked_post_submit_checker_policy_body=( + canonical_task.locked_post_submit_checker_policy_body + ), + locked_review_policy_version=canonical_task.locked_review_policy_version, + locked_revision_policy_version=( + canonical_task.locked_revision_policy_version + ), + locked_payment_policy_version=( + canonical_task.locked_payment_policy_version + ), + locked_guide_source_snapshot_id=( + canonical_task.locked_guide_source_snapshot_id + ), + locked_guide_source_snapshot_hash=( + canonical_task.locked_guide_source_snapshot_hash + ), + locked_effective_project_submission_artifact_policy_id=( + canonical_task.locked_effective_project_submission_artifact_policy_id + ), + locked_effective_project_submission_artifact_policy_hash=( + canonical_task.locked_effective_project_submission_artifact_policy_hash + ), + locked_pre_submit_checker_policy_id=( + canonical_task.locked_pre_submit_checker_policy_id + ), + locked_pre_submit_checker_bundle_hash=( + canonical_task.locked_pre_submit_checker_bundle_hash + ), + title="Unrelated checker task", + description="Must not own the checker output.", + status="draft", + created_by="setup-actor", + ) + ) + await session.flush() + checker_run = await session.get(CheckerRun, checker_run_id) + assert checker_run is not None + checker_run.task_id = unrelated_task_id + await session.commit() + async with minted_source(tmp_path / "scratch-source", b"checker") as source: service = ArtifactAdmissionService(session, settings, namespace) + forged = context.model_copy(update={"identity_link_id": uuid4()}) with pytest.raises( ArtifactAdmissionRelationshipError, - match="logical role is invalid", + match="service identity is unavailable", ): await service.admit( CheckerOutputArtifactAdmissionRequest( - authorization_context=context, + authorization_context=forged, checker_run_id=UUID(checker_run_id), - logical_role="é" * 100, + logical_role="platform-review", source=source, ) ) @@ -1104,14 +1162,13 @@ async def test_checker_output_requires_exact_active_fixed_service_identity( assert await _count(session, ArtifactPutAttempt) == 0 await session.rollback() - forged = context.model_copy(update={"identity_link_id": uuid4()}) with pytest.raises( ArtifactAdmissionRelationshipError, - match="service identity is unavailable", + match="checker run relationship is unavailable", ): await service.admit( CheckerOutputArtifactAdmissionRequest( - authorization_context=forged, + authorization_context=context, checker_run_id=UUID(checker_run_id), logical_role="platform-review", source=source, @@ -1123,6 +1180,11 @@ async def test_checker_output_requires_exact_active_fixed_service_identity( assert await _count(session, ArtifactPutAttempt) == 0 await session.rollback() + checker_run = await session.get(CheckerRun, checker_run_id) + assert checker_run is not None + checker_run.task_id = task_id + await session.commit() + request = CheckerOutputArtifactAdmissionRequest( authorization_context=context, checker_run_id=UUID(checker_run_id), @@ -1179,6 +1241,48 @@ async def test_checker_output_requires_exact_active_fixed_service_identity( await engine.dispose() +async def test_invalid_checker_role_precedes_namespace_drift( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path) + namespace = _namespace(settings) + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + session.add( + ArtifactStorageNamespace( + id="primary", + backend=namespace.backend, + adapter=namespace.adapter, + provider_profile=namespace.provider_profile, + namespace_descriptor=namespace.namespace_descriptor, + namespace_fingerprint="sha256:" + "f" * 64, + ) + ) + await session.commit() + async with minted_source(tmp_path / "scratch-source", b"checker") as source: + with pytest.raises( + ArtifactAdmissionRelationshipError, + match="logical role is invalid", + ): + await ArtifactAdmissionService(session, settings, namespace).admit( + CheckerOutputArtifactAdmissionRequest( + authorization_context=_context(actor_kind=ActorKind.SERVICE), + checker_run_id=uuid4(), + logical_role="é" * 100, + source=source, + ) + ) + assert await _count(session, ArtifactStorageNamespace) == 1 + assert await _count(session, ArtifactAdmissionScope) == 0 + assert await _count(session, ArtifactAdmissionCharge) == 0 + assert await _count(session, ArtifactPutAttempt) == 0 + finally: + await engine.dispose() + + def test_artifact_admission_migration_preserves_prior_rows_and_round_trips_empty( isolated_database_env: str, migration_lock, From ca2b71b52c2e6c412675ea718c1bd9a8a0431150 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 07:04:25 +0100 Subject: [PATCH 06/20] Move artifact references behind provider-neutral interface --- backend/app/adapters/artifacts/local.py | 6 ++-- backend/app/adapters/artifacts/references.py | 29 ------------------- .../app/adapters/artifacts/s3_compatible.py | 6 ++-- backend/app/interfaces/artifacts.py | 22 ++++++++++++++ backend/app/modules/artifacts/service.py | 2 +- backend/tests/test_artifact_architecture.py | 20 +++++++++++++ backend/tests/test_local_artifact_store.py | 6 ++-- 7 files changed, 49 insertions(+), 42 deletions(-) delete mode 100644 backend/app/adapters/artifacts/references.py diff --git a/backend/app/adapters/artifacts/local.py b/backend/app/adapters/artifacts/local.py index e48891477..0d705344e 100644 --- a/backend/app/adapters/artifacts/local.py +++ b/backend/app/adapters/artifacts/local.py @@ -20,10 +20,6 @@ await_cancellation_resistant, run_blocking_cancellation_resistant, ) -from app.adapters.artifacts.references import ( - artifact_provider_object_ref, - parse_artifact_provider_object_ref, -) from app.core.file_locks import acquire_exclusive_file_lock from app.interfaces.artifacts import ( ARTIFACT_STORE_CAPABILITY_KEY, @@ -42,7 +38,9 @@ ArtifactStoreNamespaceClaim, ArtifactStoreNamespaceIdentity, ArtifactStoreUnavailableError, + artifact_provider_object_ref, artifact_store_namespace_material, + parse_artifact_provider_object_ref, ) from app.interfaces.external_services import ExternalServiceAdapterIdentity from app.core.hashing import canonical_json_hash diff --git a/backend/app/adapters/artifacts/references.py b/backend/app/adapters/artifacts/references.py deleted file mode 100644 index 3ac7cd5b1..000000000 --- a/backend/app/adapters/artifacts/references.py +++ /dev/null @@ -1,29 +0,0 @@ -"""Canonical provider-object references shared by artifact adapters.""" - -from __future__ import annotations - -import re - -from app.interfaces.artifacts import ArtifactOperationConflictError -from app.modules.artifacts.sources import ArtifactCommitment - - -_PROVIDER_OBJECT_REF = re.compile(r"^sha256/([0-9a-f]{2})/([0-9a-f]{62})$") - - -def artifact_provider_object_ref(commitment: ArtifactCommitment) -> str: - """Derive one identity-free reference solely from a server commitment.""" - if type(commitment) is not ArtifactCommitment: - raise ArtifactOperationConflictError("artifact commitment is invalid") - digest_hex = commitment.sha256[7:] - return f"sha256/{digest_hex[:2]}/{digest_hex[2:]}" - - -def parse_artifact_provider_object_ref(provider_object_ref: str) -> tuple[str, str]: - """Return digest path parts after enforcing the canonical grammar.""" - if not isinstance(provider_object_ref, str): - raise ArtifactOperationConflictError("artifact provider reference is invalid") - matched = _PROVIDER_OBJECT_REF.fullmatch(provider_object_ref) - if matched is None: - raise ArtifactOperationConflictError("artifact provider reference is invalid") - return matched.group(1), matched.group(2) diff --git a/backend/app/adapters/artifacts/s3_compatible.py b/backend/app/adapters/artifacts/s3_compatible.py index ff5262d32..76269d39f 100644 --- a/backend/app/adapters/artifacts/s3_compatible.py +++ b/backend/app/adapters/artifacts/s3_compatible.py @@ -46,10 +46,6 @@ get_current_datetime, ) -from app.adapters.artifacts.references import ( - artifact_provider_object_ref, - parse_artifact_provider_object_ref, -) from app.core.config import Settings from app.core.hashing import canonical_json_hash from app.core.s3_validation import ( @@ -75,7 +71,9 @@ ArtifactStoreNamespaceClaim, ArtifactStoreNamespaceIdentity, ArtifactStoreUnavailableError, + artifact_provider_object_ref, artifact_store_namespace_material, + parse_artifact_provider_object_ref, ) from app.interfaces.external_services import ExternalServiceAdapterIdentity from app.modules.artifacts.preparation import HARD_MAXIMUM_ARTIFACT_BYTES diff --git a/backend/app/interfaces/artifacts.py b/backend/app/interfaces/artifacts.py index 2dd8e11fa..ea182332e 100644 --- a/backend/app/interfaces/artifacts.py +++ b/backend/app/interfaces/artifacts.py @@ -4,6 +4,7 @@ from collections.abc import AsyncIterator from dataclasses import dataclass +import re from typing import Protocol from app.core.hashing import canonical_json_hash @@ -17,6 +18,9 @@ ARTIFACT_STORE_CAPABILITY_KEY = "artifact_store" _MAXIMUM_PROVIDER_OBJECT_REF_LENGTH = 1024 +_CANONICAL_PROVIDER_OBJECT_REF = re.compile( + r"^sha256/([0-9a-f]{2})/([0-9a-f]{62})$" +) _RESERVED_NAMESPACE_DESCRIPTOR_KEYS = frozenset( {"adapter", "backend", "provider_profile"} ) @@ -262,6 +266,24 @@ class ArtifactOperationConflictError(ArtifactStoreError): category = "conflict" +def artifact_provider_object_ref(commitment: ArtifactCommitment) -> str: + """Derive one identity-free provider reference from a commitment.""" + if type(commitment) is not ArtifactCommitment: + raise ArtifactOperationConflictError("artifact commitment is invalid") + digest_hex = commitment.sha256[7:] + return f"sha256/{digest_hex[:2]}/{digest_hex[2:]}" + + +def parse_artifact_provider_object_ref(provider_object_ref: str) -> tuple[str, str]: + """Return digest path parts after enforcing the canonical grammar.""" + if not isinstance(provider_object_ref, str): + raise ArtifactOperationConflictError("artifact provider reference is invalid") + matched = _CANONICAL_PROVIDER_OBJECT_REF.fullmatch(provider_object_ref) + if matched is None: + raise ArtifactOperationConflictError("artifact provider reference is invalid") + return matched.group(1), matched.group(2) + + class ArtifactLimitExceededError(ArtifactStoreError): """Raised when bounded artifact input exceeds its hard limit.""" diff --git a/backend/app/modules/artifacts/service.py b/backend/app/modules/artifacts/service.py index 371060ddd..256319604 100644 --- a/backend/app/modules/artifacts/service.py +++ b/backend/app/modules/artifacts/service.py @@ -7,7 +7,6 @@ from sqlalchemy.ext.asyncio import AsyncSession -from app.adapters.artifacts.references import artifact_provider_object_ref from app.core.config import Settings from app.core.hashing import canonical_json_hash from app.db.session import get_session_factory @@ -15,6 +14,7 @@ ArtifactStore, ArtifactStoreBootstrap, ArtifactStoreNamespaceClaim, + artifact_provider_object_ref, artifact_store_namespace_material, ) from app.interfaces.external_services import ExternalServiceAdapterIdentity diff --git a/backend/tests/test_artifact_architecture.py b/backend/tests/test_artifact_architecture.py index d1ba72b9e..5c8ef48ab 100644 --- a/backend/tests/test_artifact_architecture.py +++ b/backend/tests/test_artifact_architecture.py @@ -127,6 +127,26 @@ def test_artifact_domain_has_no_provider_execution_during_admission_foundation() assert violations == [] +def test_artifact_domain_does_not_import_adapter_modules() -> None: + """Keep provider-neutral artifact rules independent from adapters.""" + violations: list[str] = [] + for path in _python_files(APP_ROOT / "modules" / "artifacts"): + for node in ast.walk(_tree(path)): + if isinstance(node, ast.ImportFrom) and ( + node.module or "" + ).startswith("app.adapters.artifacts"): + violations.append( + f"{path.relative_to(BACKEND_ROOT)} imports {node.module}" + ) + if isinstance(node, ast.Import): + for alias in node.names: + if alias.name.startswith("app.adapters.artifacts"): + violations.append( + f"{path.relative_to(BACKEND_ROOT)} imports {alias.name}" + ) + assert violations == [] + + def test_concrete_adapter_construction_has_one_composition_path() -> None: factory_calls: list[Path] = [] adapter_calls: list[Path] = [] diff --git a/backend/tests/test_local_artifact_store.py b/backend/tests/test_local_artifact_store.py index 72bacc12e..30852212b 100644 --- a/backend/tests/test_local_artifact_store.py +++ b/backend/tests/test_local_artifact_store.py @@ -14,10 +14,6 @@ import pytest from app.adapters.artifacts.local import LocalStorageAdapter, LocalStorageBootstrap -from app.adapters.artifacts.references import ( - artifact_provider_object_ref, - parse_artifact_provider_object_ref, -) from app.interfaces.artifacts import ( ArtifactByteRange, ArtifactConfigurationError, @@ -26,6 +22,8 @@ ArtifactOperationConflictError, ArtifactStoreUnavailableError, ArtifactStoreNamespaceClaim, + artifact_provider_object_ref, + parse_artifact_provider_object_ref, ) from app.modules.artifacts.sources import ArtifactCommitment, CommittedArtifactSource from tests.artifact_store_helpers import ( From c477d8a7e7b8eed4c89ff348b2680ab2d9a59249 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 07:57:18 +0100 Subject: [PATCH 07/20] Revalidate admission actors and replay capacity --- backend/app/modules/artifacts/repository.py | 15 +- backend/app/modules/artifacts/service.py | 52 +++- backend/tests/test_artifact_admission.py | 308 +++++++++++++++++++- 3 files changed, 344 insertions(+), 31 deletions(-) diff --git a/backend/app/modules/artifacts/repository.py b/backend/app/modules/artifacts/repository.py index d6f07de4b..089699d43 100644 --- a/backend/app/modules/artifacts/repository.py +++ b/backend/app/modules/artifacts/repository.py @@ -64,14 +64,15 @@ class CheckerOutputAdmissionFacts: @dataclass(frozen=True, slots=True) -class ServiceActorFacts: - """Locked service profile and identity-link state.""" +class ActorAdmissionFacts: + """Locked actor profile and exact identity-link state.""" actor_profile_id: str actor_kind: str actor_status: str service_identity: str | None identity_link_id: str + identity_link_subject_kind: str identity_link_status: str @@ -209,8 +210,10 @@ async def get_checker_output_admission_facts( task_id=row.task_id, ) - async def lock_service_actor(self, actor_profile_id: str) -> ServiceActorFacts | None: - """Lock one canonical service profile and its exact identity link.""" + async def lock_admission_actor( + self, actor_profile_id: str + ) -> ActorAdmissionFacts | None: + """Lock one canonical actor profile and its exact identity link.""" row = ( await self._session.execute( select( @@ -219,6 +222,7 @@ async def lock_service_actor(self, actor_profile_id: str) -> ServiceActorFacts | ActorProfile.status.label("actor_status"), ActorProfile.service_identity, ActorIdentityLink.id.label("identity_link_id"), + ActorIdentityLink.subject_kind.label("identity_link_subject_kind"), ActorIdentityLink.status.label("identity_link_status"), ) .join(ActorIdentityLink, ActorIdentityLink.actor_profile_id == ActorProfile.id) @@ -228,12 +232,13 @@ async def lock_service_actor(self, actor_profile_id: str) -> ServiceActorFacts | ).one_or_none() if row is None: return None - return ServiceActorFacts( + return ActorAdmissionFacts( actor_profile_id=row.id, actor_kind=row.actor_kind, actor_status=row.actor_status, service_identity=row.service_identity, identity_link_id=row.identity_link_id, + identity_link_subject_kind=row.identity_link_subject_kind, identity_link_status=row.identity_link_status, ) diff --git a/backend/app/modules/artifacts/service.py b/backend/app/modules/artifacts/service.py index 256319604..6be22c4ed 100644 --- a/backend/app/modules/artifacts/service.py +++ b/backend/app/modules/artifacts/service.py @@ -214,13 +214,6 @@ async def admit( ], } ) - replay = await self._existing_attempt_result( - facts.operation_identity, - request_digest, - ) - if replay is not None: - return replay - counters = await self._repo.ensure_and_lock_admission_scopes( [ (scope.scope_type, scope.scope_id, scope.limit_bytes) @@ -230,12 +223,10 @@ async def admit( # A concurrent first caller may have committed while these shared # scope locks were pending. Recheck under serialization before any # counter or charge mutation. - replay = await self._existing_attempt_result( + replay = await self._existing_attempt( facts.operation_identity, request_digest, ) - if replay is not None: - return replay charges = await self._reserve_charges( scopes=scopes, counters=counters, @@ -243,6 +234,16 @@ async def admit( sha256=commitment.sha256, byte_count=commitment.byte_count, ) + if replay is not None: + linked_charge_ids = await self._repo.list_put_attempt_charge_ids( + replay.id + ) + reserved_charge_ids = tuple(sorted(charge.id for charge in charges)) + if linked_charge_ids != reserved_charge_ids: + raise ArtifactAdmissionConfigurationError( + "artifact admission replay charge set is incomplete" + ) + return await self._result(replay, replayed=True) database_now = await self._repo.database_now() attempt = ArtifactPutAttempt( id=str(uuid4()), @@ -323,6 +324,7 @@ async def _guide_facts( raise ArtifactAdmissionRelationshipError( "guide artifact producer must be a human actor" ) + await self._require_active_human_actor(context) item_id = str(request.guide_source_item_id) row = await self._repo.get_guide_admission_facts(item_id) commitment = request.source.commitment @@ -360,6 +362,7 @@ async def _contributor_facts( raise ArtifactAdmissionRelationshipError( "contributor artifact producer must be a human actor" ) + await self._require_active_human_actor(context) item_id = str(request.upload_item_id) row = await self._repo.get_contributor_admission_facts(item_id) commitment = request.source.commitment @@ -402,7 +405,7 @@ async def _checker_output_facts( "checker output producer must be a service actor" ) logical_role = request.logical_role - service_actor = await self._repo.lock_service_actor( + service_actor = await self._repo.lock_admission_actor( str(context.actor_profile_id) ) if ( @@ -410,6 +413,7 @@ async def _checker_output_facts( or service_actor.actor_kind != "service" or service_actor.actor_status != "active" or service_actor.identity_link_id != str(context.identity_link_id) + or service_actor.identity_link_subject_kind != "service" or service_actor.identity_link_status != "active" or service_actor.service_identity != ServiceIdentity.ARTIFACT_CHECKER_OUTPUT.value @@ -443,6 +447,24 @@ async def _checker_output_facts( operation_identity=operation_identity, ) + async def _require_active_human_actor( + self, context: AuthorizationContext + ) -> None: + """Revalidate and lock exact human identity state inside admission.""" + actor = await self._repo.lock_admission_actor(str(context.actor_profile_id)) + if ( + actor is None + or actor.actor_kind != "human" + or actor.actor_status != "active" + or actor.service_identity is not None + or actor.identity_link_id != str(context.identity_link_id) + or actor.identity_link_subject_kind != "human" + or actor.identity_link_status != "active" + ): + raise ArtifactAdmissionRelationshipError( + "artifact admission human identity is unavailable" + ) + @staticmethod def _validate_logical_role(value: str) -> str: """Require one bounded printable checker-output role.""" @@ -495,12 +517,12 @@ def _configured_limits(self) -> dict[str, int]: ) return {key: int(value) for key, value in values.items()} - async def _existing_attempt_result( + async def _existing_attempt( self, operation_identity: str, request_digest: str, - ) -> ArtifactAdmissionResult | None: - """Return an exact replay or reject changed input for one operation.""" + ) -> ArtifactPutAttempt | None: + """Load an exact replay or reject changed input for one operation.""" existing = await self._repo.get_put_attempt_by_operation(operation_identity) if existing is None: return None @@ -508,7 +530,7 @@ async def _existing_attempt_result( raise ArtifactAdmissionConflictError( "artifact admission operation input changed" ) - return await self._result(existing, replayed=True) + return existing async def _reserve_charges( self, diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index ab11b1292..fa43b9084 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -3,6 +3,7 @@ from __future__ import annotations import asyncio +from dataclasses import replace from datetime import UTC, datetime, timedelta from pathlib import Path from uuid import UUID, uuid4 @@ -144,13 +145,49 @@ def _context( ) +async def _seed_human_actor( + session, + context: AuthorizationContext, +) -> None: + """Persist the exact active human actor carried by a test context.""" + actor_profile_id = str(context.actor_profile_id) + if await session.get(ActorProfile, actor_profile_id) is not None: + return + session.add( + ActorProfile( + id=actor_profile_id, + actor_kind="human", + status="active", + provisioning_method="automatic_first_access", + service_identity=None, + created_by="test", + ) + ) + await session.flush() + session.add( + ActorIdentityLink( + id=str(context.identity_link_id), + actor_profile_id=actor_profile_id, + issuer="https://issuer.example.test", + subject=f"human-{actor_profile_id}", + subject_kind="human", + status="active", + linked_by="test", + last_verified_at=datetime.now(UTC), + ) + ) + await session.flush() + + async def _seed_guide( session, *, - captured_by: str, + context: AuthorizationContext, content_hash: str, media_type: str, ) -> tuple[str, str]: + await _seed_human_actor(session, context) + captured_by = str(context.actor_profile_id) project_id = str(uuid4()) guide_id = str(uuid4()) snapshot_id = str(uuid4()) @@ -206,9 +243,11 @@ async def _seed_guide( async def _seed_contributor_items( session, *, - actor_profile_id: str, + context: AuthorizationContext, commitments: tuple[tuple[str, int, str], ...], ) -> tuple[str, str, tuple[str, ...]]: + await _seed_human_actor(session, context) + actor_profile_id = str(context.actor_profile_id) project_id = str(uuid4()) task_id = str(uuid4()) upload_session_id = str(uuid4()) @@ -551,13 +590,13 @@ async def test_guide_admission_derives_three_scopes_without_provider_evidence( ) as source: project_id, item_id = await _seed_guide( session, - captured_by=str(context.actor_profile_id), + context=context, content_hash=source.commitment.sha256, media_type=source.commitment.media_type, ) with pytest.raises( ArtifactAdmissionRelationshipError, - match="guide source item relationship is unavailable", + match="artifact admission human identity is unavailable", ): await ArtifactAdmissionService( session, @@ -637,6 +676,125 @@ async def test_guide_admission_derives_three_scopes_without_provider_evidence( await engine.dispose() +async def test_human_admission_revalidates_exact_active_profile_and_link( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path) + namespace = _namespace(settings) + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + async with minted_source( + tmp_path / "guide-source", + b"guide", + media_type="text/markdown", + ) as guide_source: + _, guide_item_id = await _seed_guide( + session, + context=context, + content_hash=guide_source.commitment.sha256, + media_type=guide_source.commitment.media_type, + ) + async with minted_source( + tmp_path / "contributor-source", + b"work", + ) as contributor_source: + _, _, upload_item_ids = await _seed_contributor_items( + session, + context=context, + commitments=( + ( + contributor_source.commitment.sha256, + contributor_source.commitment.byte_count, + contributor_source.commitment.media_type, + ), + ), + ) + requests = ( + GuideArtifactAdmissionRequest( + authorization_context=context, + guide_source_item_id=UUID(guide_item_id), + source=guide_source, + ), + ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(upload_item_ids[0]), + source=contributor_source, + ), + ) + forged_context = context.model_copy( + update={"identity_link_id": uuid4()} + ) + for request in requests: + with pytest.raises( + ArtifactAdmissionRelationshipError, + match="artifact admission human identity is unavailable", + ): + await ArtifactAdmissionService( + session, settings, namespace + ).admit( + replace( + request, + authorization_context=forged_context, + ) + ) + + link = await session.get( + ActorIdentityLink, + str(context.identity_link_id), + ) + assert link is not None + link.status = "revoked" + link.revoked_by = "test" + link.revoked_at = datetime.now(UTC) + link.revoked_reason = "test revocation" + await session.commit() + for request in requests: + with pytest.raises( + ArtifactAdmissionRelationshipError, + match="artifact admission human identity is unavailable", + ): + await ArtifactAdmissionService( + session, settings, namespace + ).admit(request) + + link.status = "active" + link.revoked_by = None + link.revoked_at = None + link.revoked_reason = None + link.reactivated_by = "test" + link.reactivated_at = datetime.now(UTC) + link.reactivation_reason = "test reactivation" + profile = await session.get( + ActorProfile, + str(context.actor_profile_id), + ) + assert profile is not None + profile.status = "suspended" + profile.suspended_by = "test" + profile.suspended_at = datetime.now(UTC) + profile.suspension_reason = "test suspension" + await session.commit() + for request in requests: + with pytest.raises( + ArtifactAdmissionRelationshipError, + match="artifact admission human identity is unavailable", + ): + await ArtifactAdmissionService( + session, settings, namespace + ).admit(request) + + assert await _count(session, ArtifactStorageNamespace) == 0 + assert await _count(session, ArtifactAdmissionScope) == 0 + assert await _count(session, ArtifactAdmissionCharge) == 0 + assert await _count(session, ArtifactPutAttempt) == 0 + finally: + await engine.dispose() + + async def test_exact_replay_returns_one_attempt_and_one_charge_set( admission_database_env: str, tmp_path: Path, @@ -651,7 +809,7 @@ async def test_exact_replay_returns_one_attempt_and_one_charge_set( async with minted_source(tmp_path / "scratch-source", b"same") as source: _, _, item_ids = await _seed_contributor_items( session, - actor_profile_id=str(context.actor_profile_id), + context=context, commitments=( ( source.commitment.sha256, @@ -686,6 +844,134 @@ async def test_exact_replay_returns_one_attempt_and_one_charge_set( await engine.dispose() +async def test_exact_replay_reacquires_released_charges_under_capacity( + admission_database_env: str, + tmp_path: Path, +) -> None: + settings = _settings(tmp_path, maximum_bytes=4) + namespace = _namespace(settings) + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + async with minted_source(tmp_path / "first-source", b"aaaa") as first_source: + async with minted_source( + tmp_path / "second-source", b"bbbb" + ) as second_source: + first_sha256 = first_source.commitment.sha256 + second_sha256 = second_source.commitment.sha256 + _, _, item_ids = await _seed_contributor_items( + session, + context=context, + commitments=( + ( + first_source.commitment.sha256, + first_source.commitment.byte_count, + first_source.commitment.media_type, + ), + ( + second_source.commitment.sha256, + second_source.commitment.byte_count, + second_source.commitment.media_type, + ), + ), + ) + first_request = ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[0]), + source=first_source, + ) + second_request = ContributorArtifactAdmissionRequest( + authorization_context=context, + upload_item_id=UUID(item_ids[1]), + source=second_source, + ) + first = await ArtifactAdmissionService( + session, settings, namespace + ).admit(first_request) + first_attempt = await session.get( + ArtifactPutAttempt, + str(first.attempt_id), + ) + assert first_attempt is not None + first_attempt.status = "absent_replay_required" + counters = ( + await session.execute(select(ArtifactAdmissionScope)) + ).scalars().all() + first_charges = ( + await session.execute( + select(ArtifactAdmissionCharge).where( + ArtifactAdmissionCharge.sha256 + == first_sha256 + ) + ) + ).scalars().all() + released_at = datetime.now(UTC) + for charge in first_charges: + charge.state = "released" + charge.released_at = released_at + charge.cas_version += 1 + for counter in counters: + counter.counted_bytes = 0 + counter.cas_version += 1 + await session.commit() + + await ArtifactAdmissionService(session, settings, namespace).admit( + second_request + ) + with pytest.raises(ArtifactAdmissionCapacityError): + await ArtifactAdmissionService( + session, settings, namespace + ).admit(first_request) + + second_charges = ( + await session.execute( + select(ArtifactAdmissionCharge).where( + ArtifactAdmissionCharge.sha256 + == second_sha256 + ) + ) + ).scalars().all() + counters = ( + await session.execute(select(ArtifactAdmissionScope)) + ).scalars().all() + for charge in second_charges: + charge.state = "released" + charge.released_at = datetime.now(UTC) + charge.cas_version += 1 + for counter in counters: + counter.counted_bytes = 0 + counter.cas_version += 1 + await session.commit() + + replay = await ArtifactAdmissionService( + session, settings, namespace + ).admit(first_request) + + assert replay.replayed is True + assert replay.attempt_id == first.attempt_id + refreshed_first_charges = ( + await session.execute( + select(ArtifactAdmissionCharge).where( + ArtifactAdmissionCharge.sha256 + == first_sha256 + ) + ) + ).scalars().all() + refreshed_counters = ( + await session.execute(select(ArtifactAdmissionScope)) + ).scalars().all() + assert {charge.state for charge in refreshed_first_charges} == { + "provisional" + } + assert {charge.released_at for charge in refreshed_first_charges} == {None} + assert {counter.counted_bytes for counter in refreshed_counters} == {4} + assert await _count(session, ArtifactPutAttempt) == 2 + finally: + await engine.dispose() + + async def test_contributor_admission_rejects_cross_project_task_relationship( admission_database_env: str, tmp_path: Path, @@ -700,7 +986,7 @@ async def test_contributor_admission_rejects_cross_project_task_relationship( async with minted_source(tmp_path / "scratch-source", b"contributor") as source: _, _, item_ids = await _seed_contributor_items( session, - actor_profile_id=str(context.actor_profile_id), + context=context, commitments=( ( source.commitment.sha256, @@ -764,7 +1050,7 @@ async def test_same_content_distinct_operations_deduplicate_scope_bytes( ) _, _, item_ids = await _seed_contributor_items( seed_session, - actor_profile_id=str(context.actor_profile_id), + context=context, commitments=(commitment, commitment), ) @@ -816,7 +1102,7 @@ async def test_completed_charge_deduplicates_and_released_charge_is_reacquired( ) _, _, item_ids = await _seed_contributor_items( session, - actor_profile_id=str(context.actor_profile_id), + context=context, commitments=(commitment, commitment, commitment), ) service = ArtifactAdmissionService(session, settings, namespace) @@ -899,7 +1185,7 @@ async def test_concurrent_distinct_content_cannot_oversubscribe_any_scope( async with factory() as seed_session: _, _, item_ids = await _seed_contributor_items( seed_session, - actor_profile_id=str(context.actor_profile_id), + context=context, commitments=( ( first_source.commitment.sha256, @@ -963,7 +1249,7 @@ async def test_capacity_failure_rolls_back_namespace_scopes_charges_and_attempt( async with minted_source(tmp_path / "scratch-source", b"four") as source: _, _, item_ids = await _seed_contributor_items( session, - actor_profile_id=str(context.actor_profile_id), + context=context, commitments=( ( source.commitment.sha256, @@ -1007,7 +1293,7 @@ async def test_changed_input_for_existing_operation_fails_closed( async with minted_source(tmp_path / "scratch-a", b"aaaa") as first_source: _, _, item_ids = await _seed_contributor_items( session, - actor_profile_id=str(context.actor_profile_id), + context=context, commitments=( ( first_source.commitment.sha256, From dd94fb52a0f015faa7e2dfd9802c58a76cc18179 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 08:39:43 +0100 Subject: [PATCH 08/20] Lock guide admission facts and close proof gaps --- backend/app/modules/artifacts/repository.py | 3 + backend/tests/test_artifact_admission.py | 123 ++++++++++++++++++-- docs/spec_artifact_storage_service.md | 5 + 3 files changed, 121 insertions(+), 10 deletions(-) diff --git a/backend/app/modules/artifacts/repository.py b/backend/app/modules/artifacts/repository.py index 089699d43..94f16b996 100644 --- a/backend/app/modules/artifacts/repository.py +++ b/backend/app/modules/artifacts/repository.py @@ -126,6 +126,9 @@ async def get_guide_admission_facts( GuideSourceSnapshot.id == GuideSourceSnapshotItem.source_snapshot_id, ) .where(GuideSourceSnapshotItem.id == guide_source_item_id) + .with_for_update( + of=(GuideSourceSnapshotItem, GuideSourceSnapshot) + ) ) ).one_or_none() if row is None: diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index fa43b9084..0f2b08ad7 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -12,6 +12,7 @@ from alembic.config import Config import pytest from sqlalchemy import func, select, text +from sqlalchemy.exc import DBAPIError from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine from app.adapters.artifacts.local import LocalStorageAdapter, LocalStorageBootstrap @@ -32,6 +33,7 @@ ArtifactUploadItem, ArtifactUploadSession, ) +from app.modules.artifacts.repository import ArtifactRepository from app.modules.artifacts.schemas import ( CheckerOutputArtifactAdmissionRequest, ContributorArtifactAdmissionRequest, @@ -64,7 +66,7 @@ RevisionPolicy, SubmissionArtifactPolicy, ) -from app.modules.tasks.models import Submission, WorkstreamTask +from app.modules.tasks.models import AuditEvent, Submission, WorkstreamTask from tests.artifact_store_helpers import ( artifact_admission_limit_settings, minted_source, @@ -795,6 +797,65 @@ async def test_human_admission_revalidates_exact_active_profile_and_link( await engine.dispose() +async def test_guide_admission_facts_lock_snapshot_and_item( + admission_database_env: str, + tmp_path: Path, +) -> None: + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as seed_session: + _, item_id = await _seed_guide( + seed_session, + context=context, + content_hash="sha256:" + "a" * 64, + media_type="text/markdown", + ) + + async with factory() as lock_session: + async with lock_session.begin(): + facts = await ArtifactRepository( + lock_session + ).get_guide_admission_facts(item_id) + assert facts is not None + + mutations = ( + ( + "update guide_source_snapshot_items " + "set media_type = 'application/json' where id = :item_id", + {"item_id": item_id}, + ), + ( + "update guide_source_snapshots set captured_by = :captured_by " + "where id = (select source_snapshot_id " + "from guide_source_snapshot_items where id = :item_id)", + { + "captured_by": str(uuid4()), + "item_id": item_id, + }, + ), + ) + for statement, parameters in mutations: + async with factory() as mutation_session: + with pytest.raises(DBAPIError, match="lock timeout"): + async with mutation_session.begin(): + await mutation_session.execute( + text("set local lock_timeout = '200ms'") + ) + await mutation_session.execute( + text(statement), + parameters, + ) + + async with factory() as assertion_session: + assert await _count(assertion_session, ArtifactAdmissionScope) == 0 + assert await _count(assertion_session, ArtifactAdmissionCharge) == 0 + assert await _count(assertion_session, ArtifactPutAttempt) == 0 + finally: + await engine.dispose() + + async def test_exact_replay_returns_one_attempt_and_one_charge_set( admission_database_env: str, tmp_path: Path, @@ -1027,6 +1088,7 @@ async def test_contributor_admission_rejects_cross_project_task_relationship( assert await _count(session, ArtifactAdmissionScope) == 0 assert await _count(session, ArtifactAdmissionCharge) == 0 assert await _count(session, ArtifactPutAttempt) == 0 + assert await _count(session, AuditEvent) == 0 finally: await engine.dispose() @@ -1638,17 +1700,54 @@ def test_artifact_admission_migration_refuses_populated_downgrade( ) -> None: config = _alembic_config() - async def seed_scope() -> None: + async def seed_attempt_only() -> None: engine = create_async_engine(isolated_database_env) try: - async with engine.begin() as connection: - await connection.execute( + factory = async_sessionmaker(engine, expire_on_commit=False) + async with factory() as session: + context = _context() + project_id, item_id = await _seed_guide( + session, + context=context, + content_hash="sha256:" + "b" * 64, + media_type="text/markdown", + ) + namespace_fingerprint = "sha256:" + "c" * 64 + await session.execute( text( - "insert into artifact_admission_scopes " - "(scope_type,scope_id,limit_bytes,counted_bytes,cas_version) " - "values ('deployment','primary',10,0,0)" - ) + "insert into artifact_storage_namespaces " + "(id,backend,adapter,provider_profile,namespace_descriptor," + "namespace_fingerprint) values " + "('primary','local','local','local-v2','{}',:fingerprint)" + ), + {"fingerprint": namespace_fingerprint}, + ) + await session.execute( + text( + "insert into artifact_put_attempts " + "(id,producer_request_type,producer_type,producer_ref," + "project_id,guide_source_item_id,sha256,byte_count,media_type," + "storage_namespace_id,namespace_fingerprint,canonical_target," + "operation_identity,request_digest,status,next_run_at," + "execution_generation,cas_version,prepared_at) values " + "(:id,'guide','actor_profile',:producer_ref,:project_id," + ":item_id,:sha256,1,'text/markdown','primary',:fingerprint," + ":target,:operation_identity,:request_digest,'prepared',now()," + "0,0,now())" + ), + { + "id": str(uuid4()), + "producer_ref": str(context.actor_profile_id), + "project_id": project_id, + "item_id": item_id, + "sha256": "sha256:" + "b" * 64, + "fingerprint": namespace_fingerprint, + "target": "sha256/bb/" + "b" * 62, + "operation_identity": "sha256:" + "d" * 64, + "request_digest": "sha256:" + "e" * 64, + }, ) + await session.commit() finally: await engine.dispose() @@ -1657,7 +1756,11 @@ async def cleanup() -> None: try: async with engine.begin() as connection: await connection.execute( - text("truncate table artifact_admission_scopes cascade") + text( + "truncate table artifact_put_attempt_charges, " + "artifact_put_attempts, artifact_admission_charges, " + "artifact_admission_scopes cascade" + ) ) finally: await engine.dispose() @@ -1666,7 +1769,7 @@ async def cleanup() -> None: try: asyncio.run(_reset_admission_test_schema(isolated_database_env)) command.upgrade(config, "0027_artifact_admission") - asyncio.run(seed_scope()) + asyncio.run(seed_attempt_only()) with pytest.raises( RuntimeError, match="cannot downgrade populated artifact admission ledger", diff --git a/docs/spec_artifact_storage_service.md b/docs/spec_artifact_storage_service.md index 615725fe3..64b0983b7 100644 --- a/docs/spec_artifact_storage_service.md +++ b/docs/spec_artifact_storage_service.md @@ -1152,6 +1152,11 @@ Implementation is a clean cut: Operator must reprovision an empty database/storage namespace out of band and reingest authoritative bytes through v2; records whose authoritative bytes are unavailable are not migrated. +- migration `0027` installs the durable admission ledger and prepared + put-attempt tables. Its downgrade locks every owned table and refuses to + remove the foundation when any admission scope, charge, attempt, or + attempt-charge link exists; downgrade is permitted only while all four + tables are empty. Every migration proves fresh upgrade, prior-head upgrade, populated-state preservation or explicit refusal, empty downgrade/re-upgrade, and no artifact From c66ee91fdf6c153ae89772d9bca5e38290317eab Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 09:18:33 +0100 Subject: [PATCH 09/20] Record ART 02C1 internal review evidence --- ...S-ART-001-02C1-internal-review-evidence.md | 100 ++++++++++++++++++ .../WS-ART-001-02C1-pr-trust-bundle.md | 99 +++++++++++++++++ 2 files changed, 199 insertions(+) create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md new file mode 100644 index 000000000..c5e5eb313 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md @@ -0,0 +1,100 @@ +# Internal Review Evidence: WS-ART-001-02C1 + +## Chunk + +`WS-ART-001-02C1`: Admission And Put-Attempt Foundation + +open sub-agent sessions: none + +valid findings addressed: yes + +## Reviewed Revision + +Reviewed code SHA: `b4d54469b1590cf43fd9f496c64b6172577c0eec` + +Reviewed at: 2026-07-19T08:09:47Z + +Reviewer run IDs: senior-engineering=019f795c-78be-7701-b920-ac956612bf3d; QA/test=019f795c-7b83-7602-a5d2-aa19f4f4c7c9; security/auth=019f795c-81a9-7cd0-9f4c-e6adf9f4b558; product/ops=019f795c-8985-7582-b695-c528c99d4321; architecture=019f795c-9173-70e0-86a9-709afb162a52; reuse/dedup=019f795c-9ba9-7153-b7e9-4f6f900dbe21; CI-integrity=019f7966-0cb2-7343-a08e-946e26c51127; test-delta=019f7966-168e-7053-93e6-f43d184e96da; docs=019f7966-1dec-7be2-8417-c4ea9bc5329a + +The reviewed base is trusted `main` at +`93dd392484b397cfdfaaa833631dc2c27f591ed7`, including merged AUTH PR #152. +Only review artifacts and initiative status may change after the reviewed SHA. +Any implementation, test, workflow, policy, or chunk-contract change invalidates +this evidence and requires a new exact-SHA review cycle. + +## Reviewed Change + +- Added durable task, producer, project, and deployment admission scopes with + server-owned limits and unique content charges. +- Added closed guide, contributor, and checker-output admission requests whose + relationships and producer authority are resolved and locked by Workstream. +- Atomically claims the storage namespace, reserves capacity, writes audit + evidence, and creates one `prepared` `ArtifactPutAttempt` before provider I/O. +- Deduplicates exact replay while reacquiring released charges only after + capacity and linked-charge revalidation. +- Locks guide source items and snapshots during authoritative admission so the + persisted attempt cannot race mutable guide facts. +- Keeps provider execution, verification, publication, recovery, routes, and + product cutover out of scope and inactive. +- Adds migration `0027` with populated-state downgrade refusal across scopes, + charges, attempts, and attempt-charge links. + +## Reviewer Results + +| Reviewer | Result | Blocking findings | Notes | +|---|---:|---|---| +| senior engineering | PASS | None | Transaction, replay, rollback, failure, and scope behavior are maintainable and bounded. | +| QA/test | PASS | None | Real PostgreSQL concurrency, lock, rollback, relationship, and migration cases satisfy the contract. | +| security/auth | PASS | None | Exact actor, identity-link, service-identity, relationship, scope, and replay checks fail closed. | +| product/ops | PASS | None | Admission remains internal and does not alter task, review, revision, contribution, or compensation lifecycle state. | +| architecture | PASS | None | Provider-neutral boundaries and transaction ownership are preserved with no route or provider execution drift. | +| reuse/dedup | PASS | None | Canonical artifact interfaces are reused and no parallel provider-reference or admission abstraction remains. | +| CI integrity | PASS | None | The new audit coverage gate is additive; the repository 78 percent floor and cumulative 90 percent gates remain fail closed. | +| test delta | PASS | None | No skipped or weakened tests; removed direct-write tests correspond to the deliberately removed provider-finalization path. | +| docs | PASS | None | Active docs, migration behavior, terminology, scope exclusions, and links match the implementation. | + +## Valid Findings Addressed + +- Replaced read-only guide admission facts with row locks on both the exact + source item and its immutable snapshot, plus a two-transaction lock-timeout + regression test. +- Proved capacity failure leaves no scope, charge, attempt, or audit residue. +- Proved an attempt-only populated state prevents destructive migration + downgrade. +- Moved provider-object reference parsing and construction behind the canonical + provider-neutral artifact interface. +- Revalidated canonical active human profiles and identity links during guide + and contributor admission. +- Revalidated replay capacity and the exact attempt-charge set before returning + an existing attempt. +- Bound checker output to the canonical submission and task relationship. + +## Commands Run + +```bash +cd backend && WORKSTREAM_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_artifact_admission.py tests/test_artifact_architecture.py tests/test_artifact_cleanup_wiring.py tests/test_artifact_preparation.py tests/test_artifact_store_conformance.py tests/test_artifacts.py tests/test_local_artifact_store.py tests/test_s3_artifact_store.py tests/test_audit.py tests/test_config.py -q --cov=app.interfaces.artifact_operations --cov=app.modules.artifacts --cov=app.modules.audit --cov=app.core.config --cov-report=term-missing --cov-fail-under=90 +cd backend && .venv/bin/ruff check app tests +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/test_agent_gates.py +python3 scripts/check_markdown_links.py +git diff --check +``` + +Results: 369 tests passed in 650.13 seconds with 94.32 percent scoped +coverage. Ruff, the stale artifact contract scan, 88 agent-gate tests, +Markdown links, and diff integrity passed. GitHub Backend CI remains +authoritative for the isolated full repository suite and 78 percent floor. + +## Remaining Risks + +- Provider execution, provider acknowledgement verification, and publication + remain intentionally unavailable until separately approved later chunks. +- Native AWS remains runtime-ineligible pending its separately owned live proof. +- The admission service remains an internal foundation and is not yet wired to + project or submission product routes. + +## Stop Condition + +Publish this evidence-bound candidate for GitHub Actions, CodeRabbit, and +explicit human review. Do not merge without the user's approval and do not +start `WS-ART-001-02C2` automatically. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md new file mode 100644 index 000000000..a680aafcd --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md @@ -0,0 +1,99 @@ +# PR Trust Bundle: WS-ART-001-02C1 + +## Chunk + +`WS-ART-001-02C1` - Admission And Put-Attempt Foundation + +Merge intent: `.agent-loop/merge-intents/WS-ART-001-02C1.json` + +## Goal + +Create the durable admission and prepared-attempt transaction required before +any artifact provider write, while keeping execution and product cutover +inactive. + +## What Changed + +- Added server-derived task, producer, project, and deployment byte limits. +- Added unique provisional/completed/released content charges with exact replay + deduplication and concurrent oversubscription protection. +- Added closed guide, contributor, and checker-output admission commands. +- Added one atomic namespace-claim, capacity-reservation, audit, and prepared + `ArtifactPutAttempt` transaction. +- Added exact human and service identity revalidation, canonical relationship + checks, guide-source row locking, and replay charge-set validation. +- Added migration and real PostgreSQL concurrency, rollback, and downgrade + proofs. + +## Scope Control + +This chunk adds no provider write, provider observation, verification, +publication, Celery execution, recovery attempt, Operator/public route, product +cutover, task claim, reviewer lease, R2 path, or Flow Node path. + +## Acceptance Proof + +- [x] Callers cannot supply or weaken admission scopes. +- [x] Same content is charged once per applicable scope. +- [x] Concurrent reservations cannot double-charge or oversubscribe. +- [x] Reservation failure leaves no attempt, charge, audit success, or provider call. +- [x] One committed prepared attempt is required before later provider work. +- [x] Guide facts, human authority, service identity, and producer relationships are revalidated transactionally. +- [x] Provider execution fields remain inactive and no product route exposes admission. +- [x] Exactly one schema-v2 merge intent names only `02C2`, which still requires an explicit start. + +## Tests And Checks + +```text +369 focused tests PASS +Scoped changed-subsystem coverage 94.32% +Ruff PASS +Stale artifact contract scan PASS +88 agent-gate tests PASS +Markdown links PASS +git diff --check PASS +``` + +GitHub Backend CI remains authoritative for the isolated full suite and 78 +percent repository coverage floor. + +## Internal Review + +Reviewed code SHA: `b4d54469b1590cf43fd9f496c64b6172577c0eec` + +All nine required reviewer tracks passed with no unresolved findings. Every +reviewer session is closed. Run IDs and repaired findings are recorded in the +internal review evidence. + +## External Review + +External checks are separate from internal agent review and have not yet run on +this candidate. + +| Source | Status | Notes | +|---|---:|---| +| GitHub Actions | Pending | Must prove Agent Gates, Backend, and every required repository check on the published branch. | +| CodeRabbit | Pending | Review comments, resolutions, and reruns will be recorded in a separate external-review response file. | +| Human review | Pending | Only the user may approve merge. | + +## Remaining Risks + +- Admission is not yet connected to provider execution or product submission + routes; those remain later, separately approved chunks. +- Native AWS remains unavailable until its release-bound live proof. + +## Human Review Focus + +- Can a caller omit a scope, bypass a limit, or substitute producer context? +- Does every successful admission create exactly one durable prepared attempt + before any provider side effect? +- Do rollback and exact replay preserve byte accounting and audit integrity? +- Are provider execution, verification, recovery, and product routes still absent? + +## Human Merge Ownership + +- [ ] I can explain what changed and why. +- [ ] I know what could break. +- [ ] I accept the remaining risks. +- [ ] GitHub CI and external review pass. +- [ ] The user explicitly approved this PR for merge. From bcdecf477c0e06d404ec3e32be54bb7cac3e73d0 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 09:25:03 +0100 Subject: [PATCH 10/20] Fix ART 02C2 contract heading --- .../chunks/WS-ART-001-02C2-verification-publication-fencing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C2-verification-publication-fencing.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C2-verification-publication-fencing.md index 9d434824c..f929fdcf2 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C2-verification-publication-fencing.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C2-verification-publication-fencing.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-02C2 Verification Publication And Fencing +# Chunk Contract: WS-ART-001-02C2 - Verification Publication And Fencing Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 02C1 From a90e4c4da036c11f703160257c877b92eef4c039 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 09:32:53 +0100 Subject: [PATCH 11/20] Correct ART admission evidence wording --- .../reviews/WS-ART-001-02C1-internal-review-evidence.md | 5 +++-- .../reviews/WS-ART-001-02C1-pr-trust-bundle.md | 8 ++++---- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md index c5e5eb313..75fc29f02 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md @@ -28,8 +28,9 @@ this evidence and requires a new exact-SHA review cycle. server-owned limits and unique content charges. - Added closed guide, contributor, and checker-output admission requests whose relationships and producer authority are resolved and locked by Workstream. -- Atomically claims the storage namespace, reserves capacity, writes audit - evidence, and creates one `prepared` `ArtifactPutAttempt` before provider I/O. +- Atomically claims the storage namespace, reserves capacity, writes durable + admission evidence, and creates one `prepared` `ArtifactPutAttempt` before + provider I/O. - Deduplicates exact replay while reacquiring released charges only after capacity and linked-charge revalidation. - Locks guide source items and snapshots during authoritative admission so the diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md index a680aafcd..26554695f 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md @@ -18,8 +18,8 @@ inactive. - Added unique provisional/completed/released content charges with exact replay deduplication and concurrent oversubscription protection. - Added closed guide, contributor, and checker-output admission commands. -- Added one atomic namespace-claim, capacity-reservation, audit, and prepared - `ArtifactPutAttempt` transaction. +- Added one atomic namespace-claim, capacity-reservation, durable admission + evidence, and prepared `ArtifactPutAttempt` transaction. - Added exact human and service identity revalidation, canonical relationship checks, guide-source row locking, and replay charge-set validation. - Added migration and real PostgreSQL concurrency, rollback, and downgrade @@ -36,7 +36,7 @@ cutover, task claim, reviewer lease, R2 path, or Flow Node path. - [x] Callers cannot supply or weaken admission scopes. - [x] Same content is charged once per applicable scope. - [x] Concurrent reservations cannot double-charge or oversubscribe. -- [x] Reservation failure leaves no attempt, charge, audit success, or provider call. +- [x] Reservation failure leaves no attempt, charge, admission evidence, or provider call. - [x] One committed prepared attempt is required before later provider work. - [x] Guide facts, human authority, service identity, and producer relationships are revalidated transactionally. - [x] Provider execution fields remain inactive and no product route exposes admission. @@ -87,7 +87,7 @@ this candidate. - Can a caller omit a scope, bypass a limit, or substitute producer context? - Does every successful admission create exactly one durable prepared attempt before any provider side effect? -- Do rollback and exact replay preserve byte accounting and audit integrity? +- Do rollback and exact replay preserve byte accounting and admission-ledger integrity? - Are provider execution, verification, recovery, and product routes still absent? ## Human Merge Ownership From 43f8b635cda9f65311b2a9b2eaca063a57c9e783 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 09:59:59 +0100 Subject: [PATCH 12/20] Rebind ART 02C1 review evidence --- .../WS-ART-001-02C1-internal-review-evidence.md | 12 ++++++++---- .../reviews/WS-ART-001-02C1-pr-trust-bundle.md | 2 +- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md index 75fc29f02..9fabc104d 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md @@ -10,11 +10,11 @@ valid findings addressed: yes ## Reviewed Revision -Reviewed code SHA: `b4d54469b1590cf43fd9f496c64b6172577c0eec` +Reviewed code SHA: `2595f0ce0964624e8ae022d6cbee04d260826612` -Reviewed at: 2026-07-19T08:09:47Z +Reviewed at: 2026-07-19T08:58:54Z -Reviewer run IDs: senior-engineering=019f795c-78be-7701-b920-ac956612bf3d; QA/test=019f795c-7b83-7602-a5d2-aa19f4f4c7c9; security/auth=019f795c-81a9-7cd0-9f4c-e6adf9f4b558; product/ops=019f795c-8985-7582-b695-c528c99d4321; architecture=019f795c-9173-70e0-86a9-709afb162a52; reuse/dedup=019f795c-9ba9-7153-b7e9-4f6f900dbe21; CI-integrity=019f7966-0cb2-7343-a08e-946e26c51127; test-delta=019f7966-168e-7053-93e6-f43d184e96da; docs=019f7966-1dec-7be2-8417-c4ea9bc5329a +Reviewer run IDs: senior-engineering=019f797a-e6b0-7920-8f87-bb01c93fb972; QA/test=019f797a-e989-7441-9d6d-f52ca2a20276; security/auth=019f797a-ed1b-7c63-b8d6-2143a4152d07; product/ops=019f7988-7323-7851-b17d-8452f9db01c0; architecture=019f7988-7bfc-7670-b9bb-67342ee934b3; reuse/dedup=019f797b-03a8-7cd0-a5de-b8e5ae3c052b; CI-integrity=019f7988-8341-7b83-8324-11461058509a; test-delta=019f7988-908c-7873-a536-2360ebb123bf; docs=019f7988-a143-7a53-a3b6-c8a304d06c22 The reviewed base is trusted `main` at `93dd392484b397cfdfaaa833631dc2c27f591ed7`, including merged AUTH PR #152. @@ -59,7 +59,7 @@ this evidence and requires a new exact-SHA review cycle. - Replaced read-only guide admission facts with row locks on both the exact source item and its immutable snapshot, plus a two-transaction lock-timeout regression test. -- Proved capacity failure leaves no scope, charge, attempt, or audit residue. +- Proved capacity failure leaves no partial admission state. - Proved an attempt-only populated state prevents destructive migration downgrade. - Moved provider-object reference parsing and construction behind the canonical @@ -69,6 +69,10 @@ this evidence and requires a new exact-SHA review cycle. - Revalidated replay capacity and the exact attempt-charge set before returning an existing attempt. - Bound checker output to the canonical submission and task relationship. +- Corrected the declared `02C2` successor contract heading to the canonical + merge-intent grammar without starting or changing `02C2` behavior. +- Replaced inaccurate audit-event wording with the exact durable admission + evidence and admission-ledger terms used by this chunk. ## Commands Run diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md index 26554695f..fa67562be 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md @@ -59,7 +59,7 @@ percent repository coverage floor. ## Internal Review -Reviewed code SHA: `b4d54469b1590cf43fd9f496c64b6172577c0eec` +Reviewed code SHA: `2595f0ce0964624e8ae022d6cbee04d260826612` All nine required reviewer tracks passed with no unresolved findings. Every reviewer session is closed. Run IDs and repaired findings are recorded in the From 166baa9cb183f6a53505318a4a7ac714fe29c1b8 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 10:08:13 +0100 Subject: [PATCH 13/20] Record ART 02C1 external review --- ...S-ART-001-02C1-external-review-response.md | 54 +++++++++++++++++++ .../WS-ART-001-02C1-pr-trust-bundle.md | 9 ++-- 2 files changed, 59 insertions(+), 4 deletions(-) create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md new file mode 100644 index 000000000..3c649f685 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md @@ -0,0 +1,54 @@ +# External Review Response: WS-ART-001-02C1 + +## Boundary + +This file records GitHub Actions and CodeRabbit review separately from the +internal sub-agent evidence. It does not replace or modify the internal review +verdicts. + +Reviewed code SHA: `2595f0ce0964624e8ae022d6cbee04d260826612` + +Current evidence-only head before this record: `7ca05d5b4d54bcc337d107de781e6e289135c2d6` + +## GitHub Actions + +The first Agent Gates run failed closed because the already-declared `02C2` +contract heading omitted the canonical ` - ` separator required by merge-intent +validation. The merge intent was not weakened or changed to a null successor. +The contract heading was corrected, `02C2` remained proposed and inactive, and +local merge-intent validation plus all 88 Agent Gates passed. + +The replacement Agent Gates run passed. Backend is rerunning the authoritative +isolated full suite and 78 percent repository floor on the final external-review +evidence head. + +## CodeRabbit + +CodeRabbit completed with no review submission, no inline review thread, and no +actionable comment. + +Its pre-merge summary raised two warnings: + +1. The generic docstring calculation reported 54.37 percent. No code change was + made for this warning because the repository's authoritative configured + command, `docstr-coverage --config .docstr.yaml`, passed at 90.4 percent + against the enforced 80 percent floor. +2. The original PR description did not reproduce every trust-bundle section. + This was valid metadata feedback. The PR description now includes the chunk + and merge intent, approved intent, design, alternatives, scope, evidence, + acceptance proof, test delta, reviewer table, CI integrity, risks, follow-up, + human focus, and merge-ownership checklist. + +## Current Status + +| Source | Status | Notes | +|---|---:|---| +| CodeRabbit | PASS | No actionable review threads; both warnings were assessed and the valid metadata warning was fixed. | +| Agent Gates | PASS | Canonical successor validation and all repository agent gates pass. | +| Backend | Pending | Final evidence-only head must complete the isolated full suite and coverage gates. | +| Human review | Pending | Only the user may approve merge. | + +## Stop Condition + +Wait for the final GitHub Backend result and explicit human review. Do not +merge and do not start `WS-ART-001-02C2` automatically. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md index fa67562be..5229ffb8e 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md @@ -67,13 +67,14 @@ internal review evidence. ## External Review -External checks are separate from internal agent review and have not yet run on -this candidate. +External checks are separate from internal agent review. Their detailed status +and response are recorded in +`.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md`. | Source | Status | Notes | |---|---:|---| -| GitHub Actions | Pending | Must prove Agent Gates, Backend, and every required repository check on the published branch. | -| CodeRabbit | Pending | Review comments, resolutions, and reruns will be recorded in a separate external-review response file. | +| GitHub Actions | Running | Replacement Agent Gates passed; Backend must complete on the final evidence-only head. | +| CodeRabbit | PASS | No actionable review thread; the valid PR-description warning was fixed and the repository docstring gate independently passes. | | Human review | Pending | Only the user may approve merge. | ## Remaining Risks From 1d4f2bb652d49d9f421a5f84790e920511b024f8 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 12:26:06 +0100 Subject: [PATCH 14/20] Rebase artifact admission after contributor schema --- ...7_artifact_admission.py => 0028_artifact_admission.py} | 8 ++++---- backend/tests/test_artifact_admission.py | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-) rename backend/alembic/versions/{0027_artifact_admission.py => 0028_artifact_admission.py} (98%) diff --git a/backend/alembic/versions/0027_artifact_admission.py b/backend/alembic/versions/0028_artifact_admission.py similarity index 98% rename from backend/alembic/versions/0027_artifact_admission.py rename to backend/alembic/versions/0028_artifact_admission.py index e01604434..6a54b9f3f 100644 --- a/backend/alembic/versions/0027_artifact_admission.py +++ b/backend/alembic/versions/0028_artifact_admission.py @@ -1,7 +1,7 @@ """add durable-byte admission and prepared put attempts -Revision ID: 0027_artifact_admission -Revises: 0026_actor_profile_lifecycle +Revision ID: 0028_artifact_admission +Revises: 0027_contributor_foundation Create Date: 2026-07-19 """ @@ -11,8 +11,8 @@ import sqlalchemy as sa -revision = "0027_artifact_admission" -down_revision = "0026_actor_profile_lifecycle" +revision = "0028_artifact_admission" +down_revision = "0027_contributor_foundation" branch_labels = depends_on = None _ADMISSION_TABLES = ( diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index 0f2b08ad7..64e4c485c 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -1683,11 +1683,11 @@ async def cleanup() -> None: asyncio.run(_reset_admission_test_schema(isolated_database_env)) command.upgrade(config, "0026_actor_profile_lifecycle") asyncio.run(seed_prior_namespace()) - command.upgrade(config, "0027_artifact_admission") + command.upgrade(config, "0028_artifact_admission") assert asyncio.run(state()) == (1, True) command.downgrade(config, "0026_actor_profile_lifecycle") assert asyncio.run(state()) == (1, False) - command.upgrade(config, "0027_artifact_admission") + command.upgrade(config, "0028_artifact_admission") assert asyncio.run(state()) == (1, True) asyncio.run(cleanup()) finally: @@ -1768,7 +1768,7 @@ async def cleanup() -> None: with migration_lock(): try: asyncio.run(_reset_admission_test_schema(isolated_database_env)) - command.upgrade(config, "0027_artifact_admission") + command.upgrade(config, "0028_artifact_admission") asyncio.run(seed_attempt_only()) with pytest.raises( RuntimeError, From a2e35667f033fdab3ade698481b91de7d66e5173 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 12:31:44 +0100 Subject: [PATCH 15/20] Preserve parallel initiative loop state --- .agent-loop/LOOP_STATE.md | 14 ++++++++++---- .agent-loop/WORK_QUEUE.md | 3 ++- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index 43403e3ec..77158c3e2 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -37,10 +37,16 @@ - PR #122 merged the first automated post-merge memory implementation as `fc89fb6`; its schema-v1 cross-initiative next pointer is superseded by the schema-v2 initiative-local clean cut. -- Current ART gate: repair the Backend migration-head assertions found by PR - #154 CI, rerun deterministic proof and all nine exact-SHA internal reviewer - tracks, then return to external checks and explicit human review. No later - ART chunk starts automatically. +- Active implementation chunk: `WS-AUTH-001-CONTRIBUTOR-FOUNDATION`, explicitly + started by the user on 2026-07-19 from trusted `main` at `93dd392`. Current + exact contract `2a21166d` passed required L1 preimplementation review; + initial findings are repaired, and exact code SHA `4d1fc507` passed all nine + required internal tracks. PR publication and external checks are the current + gate; Backend must still prove 78/90 percent aggregate coverage. It changes no action + availability, and no service caller becomes executable before AUTH-09E. +- Current ART gate: integrate trusted `main`, complete deterministic 02C1 + proof, and pass all nine exact-SHA internal reviewer tracks before opening + the ART PR. No later ART chunk starts automatically. - Scope checkpoint: AWS S3 is the only v0.1 production provider; MinIO is local/CI S3 protocol proof; LocalStorage is focused development/test; R2 and Flow Node are deferred. Product modules receive narrow artifact capabilities, diff --git a/.agent-loop/WORK_QUEUE.md b/.agent-loop/WORK_QUEUE.md index 89d1efb03..2ece0c24b 100644 --- a/.agent-loop/WORK_QUEUE.md +++ b/.agent-loop/WORK_QUEUE.md @@ -4,7 +4,8 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| -| `WS-ART-001-02C1` | Admission And Put-Attempt Foundation | L1 | PR #154 open; current-main integration and Backend migration-head CI repair in progress | +| `WS-AUTH-001-CONTRIBUTOR-FOUNDATION` | Contributor Fields And Canonical-Human Lineage | L1 | Internal review passed at `4d1fc507`; PR/external checks pending; aggregate coverage mandatory in Backend | +| `WS-ART-001-02C1` | Admission And Put-Attempt Foundation | L1 | Active after PR #151 and explicit user start; implementation and deterministic proof in progress | Live post-merge state remains read from signed `automation/loop-memory` output. This authored queue records the separately approved parallel chunks. From a6f8c56256a0df70a240731b61bbec0ab4063a28 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 12:40:47 +0100 Subject: [PATCH 16/20] Update artifact admission fixture for contributors --- backend/tests/test_artifact_admission.py | 28 +++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index 64e4c485c..aa720c786 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -333,6 +333,8 @@ async def _seed_checker_output_relationships(session) -> tuple[str, str, str]: post_submit_policy_id = str(uuid4()) task_id = str(uuid4()) submission_id = str(uuid4()) + contributor_id = str(uuid4()) + contributor_link_id = str(uuid4()) checker_run_id = str(uuid4()) guide_version = "v1" snapshot_hash = canonical_json_hash({"items": []}) @@ -507,11 +509,35 @@ async def _seed_checker_output_relationships(session) -> tuple[str, str, str]: ) ) await session.flush() + session.add( + ActorProfile( + id=contributor_id, + actor_kind="human", + status="active", + provisioning_method="automatic_first_access", + service_identity=None, + created_by="test", + ) + ) + await session.flush() + session.add( + ActorIdentityLink( + id=contributor_link_id, + actor_profile_id=contributor_id, + issuer="https://issuer.example.test", + subject=f"human-{contributor_id}", + subject_kind="human", + status="active", + linked_by="test", + last_verified_at=now, + ) + ) + await session.flush() session.add( Submission( id=submission_id, task_id=task_id, - worker_id=str(uuid4()), + contributor_id=contributor_id, version=1, status="submitted", summary="Checker source submission", From 6392825f2c42d88ad5be4151454e0bfc08139f83 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 15:49:37 +0100 Subject: [PATCH 17/20] Respect actor ownership in artifact admission --- ...1-02C1-admission-put-attempt-foundation.md | 6 ++ .../versions/0028_artifact_admission.py | 8 +-- backend/app/modules/actors/repository.py | 14 +++++ backend/app/modules/actors/service.py | 36 +++++++++++ backend/app/modules/artifacts/models.py | 9 ++- backend/app/modules/artifacts/repository.py | 46 -------------- backend/app/modules/artifacts/service.py | 14 +++-- backend/tests/test_artifact_admission.py | 62 ++++++++++++++++++- backend/tests/test_artifact_architecture.py | 15 +++++ docs/spec_artifact_storage_service.md | 2 +- 10 files changed, 150 insertions(+), 62 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C1-admission-put-attempt-foundation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C1-admission-put-attempt-foundation.md index 310580a9e..9ce5fc1c8 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C1-admission-put-attempt-foundation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-02C1-admission-put-attempt-foundation.md @@ -15,9 +15,13 @@ publication, recovery, Operator routes, and product cutovers inactive. - one artifact-foundation migration; - artifact admission and put-attempt models, schemas, repository, service, and contracts; +- the actors-owned frozen admission-proof contract, repository lock, and + service method required to revalidate an exact profile/link pair in the + caller-owned admission transaction; - `backend/app/core/config.py` for durable byte limits; - generic audit repository only when existing audit support is insufficient; - focused PostgreSQL admission, concurrency, migration, and state tests; +- focused actor/artifact ownership-boundary and transactional proof tests; - `.github/workflows/backend.yml` only to expand the exact 90 percent scoped gate; - `scripts/test_agent_gates.py` only to assert that backend CI retains this chunk's exact scoped coverage sources and fail-closed 90 percent threshold; @@ -31,6 +35,8 @@ publication, recovery, Operator routes, and product cutovers inactive. - provider mutation replay, overwrite, delete, retain, or release; - task-claim or reviewer-lease changes; - production dispatch or activation. +- AUTH permission decisions or action activation, actor provisioning or + lifecycle mutation, and actor-facing routes or product cutover. ## Acceptance Criteria diff --git a/backend/alembic/versions/0028_artifact_admission.py b/backend/alembic/versions/0028_artifact_admission.py index 6a54b9f3f..c51a49a0b 100644 --- a/backend/alembic/versions/0028_artifact_admission.py +++ b/backend/alembic/versions/0028_artifact_admission.py @@ -141,7 +141,7 @@ def upgrade() -> None: name="completed_timestamp", ), sa.CheckConstraint( - "state != 'released' or released_at is not null", + "(state = 'released') = (released_at is not null)", name="released_timestamp", ), sa.ForeignKeyConstraint( @@ -187,8 +187,7 @@ def upgrade() -> None: sa.Column( "next_run_at", sa.DateTime(timezone=True), - nullable=False, - server_default=sa.func.now(), + nullable=True, ), sa.Column("executor_id", sa.String(36), nullable=True), sa.Column("lease_expires_at", sa.DateTime(timezone=True), nullable=True), @@ -263,7 +262,8 @@ def upgrade() -> None: name="versions_nonnegative", ), sa.CheckConstraint( - "status != 'prepared' or (executor_id is null and lease_expires_at is null " + "status != 'prepared' or (next_run_at is null and executor_id is null " + "and lease_expires_at is null " "and execution_generation = 0 and terminal_result_code is null " "and terminal_at is null and replica_id is null and receipt_id is null)", name="prepared_execution_inactive", diff --git a/backend/app/modules/actors/repository.py b/backend/app/modules/actors/repository.py index 6bad7db1c..487c707d6 100644 --- a/backend/app/modules/actors/repository.py +++ b/backend/app/modules/actors/repository.py @@ -92,6 +92,20 @@ async def get_actor_profile( query = query.with_for_update() return await self._session.scalar(query.execution_options(populate_existing=True)) + async def lock_exact_actor_identity( + self, + actor_profile_id: str, + identity_link_id: str, + ) -> tuple[ActorProfile, ActorIdentityLink] | None: + """Lock an exact canonical profile then its requested identity link.""" + profile = await self.get_actor_profile(actor_profile_id, for_update=True) + if profile is None: + return None + link = await self.get_identity_link_by_id(identity_link_id, for_update=True) + if link is None or link.actor_profile_id != actor_profile_id: + return None + return profile, link + async def get_service_actor( self, service_identity: str, diff --git a/backend/app/modules/actors/service.py b/backend/app/modules/actors/service.py index f22daf460..17fcfc078 100644 --- a/backend/app/modules/actors/service.py +++ b/backend/app/modules/actors/service.py @@ -96,6 +96,19 @@ class ResolvedActor: identity_link: ActorIdentityLink +@dataclass(frozen=True, slots=True) +class ActorAdmissionProof: + """Primitive canonical actor/link state locked for a caller transaction.""" + + actor_profile_id: str + actor_kind: str + actor_status: str + service_identity: str | None + identity_link_id: str + identity_link_subject_kind: str + identity_link_status: str + + class ActorService: """Resolve canonical actors and own first-human provisioning transactions.""" @@ -105,6 +118,29 @@ def __init__(self, session: AsyncSession) -> None: self._audit = AuditService(session) self._legacy_audit = AuditRepository(session) + async def lock_admission_proof( + self, + actor_profile_id: UUID, + identity_link_id: UUID, + ) -> ActorAdmissionProof | None: + """Lock and project one exact actor/link pair without deciding authority.""" + locked = await self._repo.lock_exact_actor_identity( + str(actor_profile_id), + str(identity_link_id), + ) + if locked is None: + return None + profile, link = locked + return ActorAdmissionProof( + actor_profile_id=profile.id, + actor_kind=profile.actor_kind, + actor_status=profile.status, + service_identity=profile.service_identity, + identity_link_id=link.id, + identity_link_subject_kind=link.subject_kind, + identity_link_status=link.status, + ) + async def find_verified_actor(self, token: VerifiedIssuerToken) -> ResolvedActor | None: """Return a canonical actor for an existing exact identity link.""" resolved = await self.find_actor_for_authorization(token) diff --git a/backend/app/modules/artifacts/models.py b/backend/app/modules/artifacts/models.py index 6225b279b..0f005a698 100644 --- a/backend/app/modules/artifacts/models.py +++ b/backend/app/modules/artifacts/models.py @@ -302,7 +302,7 @@ class ArtifactAdmissionCharge(Base): name="completed_timestamp", ), CheckConstraint( - "state != 'released' or released_at is not null", + "(state = 'released') = (released_at is not null)", name="released_timestamp", ), ) @@ -386,7 +386,8 @@ class ArtifactPutAttempt(Base): name="versions_nonnegative", ), CheckConstraint( - "status != 'prepared' or (executor_id is null and lease_expires_at is null " + "status != 'prepared' or (next_run_at is null and executor_id is null " + "and lease_expires_at is null " "and execution_generation = 0 and terminal_result_code is null " "and terminal_at is null and replica_id is null and receipt_id is null)", name="prepared_execution_inactive", @@ -434,9 +435,7 @@ class ArtifactPutAttempt(Base): operation_identity: Mapped[str] = mapped_column(String(71), nullable=False) request_digest: Mapped[str] = mapped_column(String(71), nullable=False) status: Mapped[str] = mapped_column(String(40), nullable=False, default="prepared", index=True) - next_run_at: Mapped[datetime] = mapped_column( - DateTime(timezone=True), nullable=False, server_default=func.now(), index=True - ) + next_run_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True) executor_id: Mapped[str | None] = mapped_column(String(36)) lease_expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) execution_generation: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) diff --git a/backend/app/modules/artifacts/repository.py b/backend/app/modules/artifacts/repository.py index 94f16b996..3a4028ac7 100644 --- a/backend/app/modules/artifacts/repository.py +++ b/backend/app/modules/artifacts/repository.py @@ -10,7 +10,6 @@ from sqlalchemy.dialects.postgresql import insert from sqlalchemy.ext.asyncio import AsyncSession -from app.modules.actors.models import ActorIdentityLink, ActorProfile from app.modules.artifacts.models import ( ArtifactAdmissionCharge, ArtifactAdmissionScope, @@ -63,19 +62,6 @@ class CheckerOutputAdmissionFacts: task_id: str -@dataclass(frozen=True, slots=True) -class ActorAdmissionFacts: - """Locked actor profile and exact identity-link state.""" - - actor_profile_id: str - actor_kind: str - actor_status: str - service_identity: str | None - identity_link_id: str - identity_link_subject_kind: str - identity_link_status: str - - class ArtifactRepository: """Persist artifact state transitions under caller-owned transactions.""" @@ -213,38 +199,6 @@ async def get_checker_output_admission_facts( task_id=row.task_id, ) - async def lock_admission_actor( - self, actor_profile_id: str - ) -> ActorAdmissionFacts | None: - """Lock one canonical actor profile and its exact identity link.""" - row = ( - await self._session.execute( - select( - ActorProfile.id, - ActorProfile.actor_kind, - ActorProfile.status.label("actor_status"), - ActorProfile.service_identity, - ActorIdentityLink.id.label("identity_link_id"), - ActorIdentityLink.subject_kind.label("identity_link_subject_kind"), - ActorIdentityLink.status.label("identity_link_status"), - ) - .join(ActorIdentityLink, ActorIdentityLink.actor_profile_id == ActorProfile.id) - .where(ActorProfile.id == actor_profile_id) - .with_for_update(of=(ActorProfile, ActorIdentityLink)) - ) - ).one_or_none() - if row is None: - return None - return ActorAdmissionFacts( - actor_profile_id=row.id, - actor_kind=row.actor_kind, - actor_status=row.actor_status, - service_identity=row.service_identity, - identity_link_id=row.identity_link_id, - identity_link_subject_kind=row.identity_link_subject_kind, - identity_link_status=row.identity_link_status, - ) - async def ensure_and_lock_admission_scopes( self, scopes: Sequence[tuple[str, str, int]], diff --git a/backend/app/modules/artifacts/service.py b/backend/app/modules/artifacts/service.py index 6be22c4ed..dd5297b94 100644 --- a/backend/app/modules/artifacts/service.py +++ b/backend/app/modules/artifacts/service.py @@ -18,6 +18,7 @@ artifact_store_namespace_material, ) from app.interfaces.external_services import ExternalServiceAdapterIdentity +from app.modules.actors.service import ActorService from app.modules.actors.service_identities import ServiceIdentity from app.modules.artifacts.models import ( ArtifactAdmissionCharge, @@ -173,6 +174,7 @@ def __init__( self._settings = settings self._namespace = namespace self._repo = ArtifactRepository(session) + self._actors = ActorService(session) async def admit( self, @@ -265,7 +267,7 @@ async def admit( operation_identity=facts.operation_identity, request_digest=request_digest, status="prepared", - next_run_at=database_now, + next_run_at=None, executor_id=None, lease_expires_at=None, execution_generation=0, @@ -405,8 +407,9 @@ async def _checker_output_facts( "checker output producer must be a service actor" ) logical_role = request.logical_role - service_actor = await self._repo.lock_admission_actor( - str(context.actor_profile_id) + service_actor = await self._actors.lock_admission_proof( + context.actor_profile_id, + context.identity_link_id, ) if ( service_actor is None @@ -451,7 +454,10 @@ async def _require_active_human_actor( self, context: AuthorizationContext ) -> None: """Revalidate and lock exact human identity state inside admission.""" - actor = await self._repo.lock_admission_actor(str(context.actor_profile_id)) + actor = await self._actors.lock_admission_proof( + context.actor_profile_id, + context.identity_link_id, + ) if ( actor is None or actor.actor_kind != "human" diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index aa720c786..efe5a73d5 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -19,6 +19,7 @@ from app.core.config import Settings from app.core.hashing import canonical_json_hash from app.modules.actors.models import ActorIdentityLink, ActorProfile +from app.modules.actors.service import ActorService from app.modules.actors.service_identities import ServiceIdentity from app.modules.checkers.models import CheckerRun from app.modules.artifacts.models import ( @@ -689,6 +690,7 @@ async def test_guide_admission_derives_three_scopes_without_provider_evidence( assert attempt.task_id is None assert attempt.executor_id is None assert attempt.lease_expires_at is None + assert attempt.next_run_at is None assert attempt.execution_generation == 0 assert {scope.scope_type for scope in scopes} == { "deployment", @@ -882,6 +884,52 @@ async def test_guide_admission_facts_lock_snapshot_and_item( await engine.dispose() +async def test_actor_admission_proof_locks_exact_profile_then_link( + admission_database_env: str, +) -> None: + context = _context() + engine = create_async_engine(admission_database_env) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as seed_session: + await _seed_human_actor(seed_session, context) + await seed_session.commit() + + async with factory() as lock_session: + async with lock_session.begin(): + proof = await ActorService(lock_session).lock_admission_proof( + context.actor_profile_id, + context.identity_link_id, + ) + assert proof is not None + assert proof.actor_profile_id == str(context.actor_profile_id) + assert proof.identity_link_id == str(context.identity_link_id) + + mutations = ( + ( + "update actor_profiles set status = 'suspended' where id = :id", + str(context.actor_profile_id), + ), + ( + "update actor_identity_links set status = 'revoked' where id = :id", + str(context.identity_link_id), + ), + ) + for statement, row_id in mutations: + async with factory() as mutation_session: + with pytest.raises(DBAPIError, match="lock timeout"): + async with mutation_session.begin(): + await mutation_session.execute( + text("set local lock_timeout = '200ms'") + ) + await mutation_session.execute( + text(statement), + {"id": row_id}, + ) + finally: + await engine.dispose() + + async def test_exact_replay_returns_one_attempt_and_one_charge_set( admission_database_env: str, tmp_path: Path, @@ -1289,6 +1337,12 @@ async def test_concurrent_distinct_content_cannot_oversubscribe_any_scope( ) async def admit(item_id: str, source): + async with ready_lock: + nonlocal ready_count + ready_count += 1 + if ready_count == 2: + start.set() + await start.wait() async with factory() as session: return await ArtifactAdmissionService( session, @@ -1302,6 +1356,9 @@ async def admit(item_id: str, source): ) ) + ready_count = 0 + ready_lock = asyncio.Lock() + start = asyncio.Event() outcomes = await asyncio.gather( admit(item_ids[0], first_source), admit(item_ids[1], second_source), @@ -1597,6 +1654,7 @@ async def test_checker_output_requires_exact_active_fixed_service_identity( assert attempt.logical_role == "platform-review" assert attempt.executor_id is None assert attempt.lease_expires_at is None + assert attempt.next_run_at is None assert attempt.execution_generation == 0 assert {scope.scope_type for scope in scopes} == { "deployment", @@ -1754,11 +1812,11 @@ async def seed_attempt_only() -> None: "(id,producer_request_type,producer_type,producer_ref," "project_id,guide_source_item_id,sha256,byte_count,media_type," "storage_namespace_id,namespace_fingerprint,canonical_target," - "operation_identity,request_digest,status,next_run_at," + "operation_identity,request_digest,status," "execution_generation,cas_version,prepared_at) values " "(:id,'guide','actor_profile',:producer_ref,:project_id," ":item_id,:sha256,1,'text/markdown','primary',:fingerprint," - ":target,:operation_identity,:request_digest,'prepared',now()," + ":target,:operation_identity,:request_digest,'prepared'," "0,0,now())" ), { diff --git a/backend/tests/test_artifact_architecture.py b/backend/tests/test_artifact_architecture.py index 5c8ef48ab..ea389a79c 100644 --- a/backend/tests/test_artifact_architecture.py +++ b/backend/tests/test_artifact_architecture.py @@ -339,3 +339,18 @@ def test_scratch_cleanup_worker_has_no_product_or_database_state() -> None: assert not any( module.startswith(forbidden_import_prefixes) for module in imported_modules ) + + +def test_artifact_repository_does_not_own_actor_persistence() -> None: + """Keep canonical actor models and queries behind the actors-owned proof API.""" + path = APP_ROOT / "modules" / "artifacts" / "repository.py" + tree = _tree(path) + imported_modules: set[str] = set() + for node in ast.walk(tree): + if isinstance(node, ast.ImportFrom) and node.module is not None: + imported_modules.add(node.module) + elif isinstance(node, ast.Import): + imported_modules.update(alias.name for alias in node.names) + assert not any(module.startswith("app.modules.actors") for module in imported_modules) + assert "actor_profiles" not in path.read_text() + assert "actor_identity_links" not in path.read_text() diff --git a/docs/spec_artifact_storage_service.md b/docs/spec_artifact_storage_service.md index 64b0983b7..9f73bad0f 100644 --- a/docs/spec_artifact_storage_service.md +++ b/docs/spec_artifact_storage_service.md @@ -1152,7 +1152,7 @@ Implementation is a clean cut: Operator must reprovision an empty database/storage namespace out of band and reingest authoritative bytes through v2; records whose authoritative bytes are unavailable are not migrated. -- migration `0027` installs the durable admission ledger and prepared +- migration `0028_artifact_admission` installs the durable admission ledger and prepared put-attempt tables. Its downgrade locks every owned table and refuses to remove the foundation when any admission scope, charge, attempt, or attempt-charge link exists; downgrade is permitted only while all four From 535069cfb1a7312d731bb14a6023ceb0894402e9 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 16:00:05 +0100 Subject: [PATCH 18/20] Prove artifact admission ledger concurrency --- backend/tests/test_artifact_admission.py | 129 +++++++++++++++++------ 1 file changed, 94 insertions(+), 35 deletions(-) diff --git a/backend/tests/test_artifact_admission.py b/backend/tests/test_artifact_admission.py index efe5a73d5..975a878d7 100644 --- a/backend/tests/test_artifact_admission.py +++ b/backend/tests/test_artifact_admission.py @@ -1170,12 +1170,32 @@ async def test_contributor_admission_rejects_cross_project_task_relationship( async def test_same_content_distinct_operations_deduplicate_scope_bytes( admission_database_env: str, tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, ) -> None: settings = _settings(tmp_path) namespace = _namespace(settings) - context = _context() + contexts = (_context(), _context()) engine = create_async_engine(admission_database_env) factory = async_sessionmaker(engine, expire_on_commit=False) + original_reserve = ArtifactRepository.ensure_and_lock_admission_scopes + ready_count = 0 + ready_lock = asyncio.Lock() + start = asyncio.Event() + + async def synchronized_reserve(repository, scopes): + nonlocal ready_count + async with ready_lock: + ready_count += 1 + if ready_count == 2: + start.set() + await start.wait() + return await original_reserve(repository, scopes) + + monkeypatch.setattr( + ArtifactRepository, + "ensure_and_lock_admission_scopes", + synchronized_reserve, + ) try: async with minted_source(tmp_path / "scratch-source", b"same") as source: async with factory() as seed_session: @@ -1184,13 +1204,27 @@ async def test_same_content_distinct_operations_deduplicate_scope_bytes( source.commitment.byte_count, source.commitment.media_type, ) - _, _, item_ids = await _seed_contributor_items( - seed_session, - context=context, - commitments=(commitment, commitment), + item_ids = [] + for context in contexts: + _, _, context_item_ids = await _seed_contributor_items( + seed_session, + context=context, + commitments=(commitment,), + ) + item_ids.append(context_item_ids[0]) + seed_session.add( + ArtifactStorageNamespace( + id="primary", + backend=namespace.backend, + adapter=namespace.adapter, + provider_profile=namespace.provider_profile, + namespace_descriptor=namespace.namespace_descriptor, + namespace_fingerprint=namespace.namespace_fingerprint, + ) ) + await seed_session.commit() - async def admit(item_id: str): + async def admit(item_id: str, context: AuthorizationContext): async with factory() as session: return await ArtifactAdmissionService( session, @@ -1204,7 +1238,9 @@ async def admit(item_id: str): ) ) - results = await asyncio.gather(*(admit(item_id) for item_id in item_ids)) + results = await asyncio.gather( + *(admit(item_id, context) for item_id, context in zip(item_ids, contexts)) + ) async with factory() as session: assert all(result.replayed is False for result in results) @@ -1212,7 +1248,7 @@ async def admit(item_id: str): await session.execute(select(ArtifactAdmissionScope)) ).scalars().all() assert {counter.counted_bytes for counter in counters} == {4} - assert await _count(session, ArtifactAdmissionCharge) == 4 + assert await _count(session, ArtifactAdmissionCharge) == 7 assert await _count(session, ArtifactPutAttempt) == 2 assert await _count(session, ArtifactPutAttemptCharge) == 8 finally: @@ -1309,40 +1345,66 @@ async def test_completed_charge_deduplicates_and_released_charge_is_reacquired( async def test_concurrent_distinct_content_cannot_oversubscribe_any_scope( admission_database_env: str, tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, ) -> None: settings = _settings(tmp_path, maximum_bytes=6) namespace = _namespace(settings) - context = _context() + contexts = (_context(), _context()) engine = create_async_engine(admission_database_env) factory = async_sessionmaker(engine, expire_on_commit=False) + original_reserve = ArtifactRepository.ensure_and_lock_admission_scopes + ready_count = 0 + ready_lock = asyncio.Lock() + start = asyncio.Event() + + async def synchronized_reserve(repository, scopes): + nonlocal ready_count + async with ready_lock: + ready_count += 1 + if ready_count == 2: + start.set() + await start.wait() + return await original_reserve(repository, scopes) + + monkeypatch.setattr( + ArtifactRepository, + "ensure_and_lock_admission_scopes", + synchronized_reserve, + ) try: async with minted_source(tmp_path / "scratch-a", b"aaaa") as first_source: async with minted_source(tmp_path / "scratch-b", b"bbbb") as second_source: async with factory() as seed_session: - _, _, item_ids = await _seed_contributor_items( - seed_session, - context=context, - commitments=( - ( - first_source.commitment.sha256, - first_source.commitment.byte_count, - first_source.commitment.media_type, - ), - ( - second_source.commitment.sha256, - second_source.commitment.byte_count, - second_source.commitment.media_type, + item_ids = [] + for context, source in zip( + contexts, + (first_source, second_source), + ): + _, _, context_item_ids = await _seed_contributor_items( + seed_session, + context=context, + commitments=( + ( + source.commitment.sha256, + source.commitment.byte_count, + source.commitment.media_type, + ), ), - ), + ) + item_ids.append(context_item_ids[0]) + seed_session.add( + ArtifactStorageNamespace( + id="primary", + backend=namespace.backend, + adapter=namespace.adapter, + provider_profile=namespace.provider_profile, + namespace_descriptor=namespace.namespace_descriptor, + namespace_fingerprint=namespace.namespace_fingerprint, + ) ) + await seed_session.commit() - async def admit(item_id: str, source): - async with ready_lock: - nonlocal ready_count - ready_count += 1 - if ready_count == 2: - start.set() - await start.wait() + async def admit(item_id: str, source, context: AuthorizationContext): async with factory() as session: return await ArtifactAdmissionService( session, @@ -1356,12 +1418,9 @@ async def admit(item_id: str, source): ) ) - ready_count = 0 - ready_lock = asyncio.Lock() - start = asyncio.Event() outcomes = await asyncio.gather( - admit(item_ids[0], first_source), - admit(item_ids[1], second_source), + admit(item_ids[0], first_source, contexts[0]), + admit(item_ids[1], second_source, contexts[1]), return_exceptions=True, ) From a93be2ec25689e1f8e036321d8a45b1fe35455ed Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 16:25:59 +0100 Subject: [PATCH 19/20] Record final ART 02C1 review evidence --- .agent-loop/LOOP_STATE.md | 6 +- .../STATUS.md | 22 ++- ...S-ART-001-02C1-external-review-response.md | 56 +++--- ...S-ART-001-02C1-internal-review-evidence.md | 161 +++++++++--------- .../WS-ART-001-02C1-pr-trust-bundle.md | 116 +++++++------ 5 files changed, 185 insertions(+), 176 deletions(-) diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index 77158c3e2..18cc796b3 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -45,8 +45,10 @@ gate; Backend must still prove 78/90 percent aggregate coverage. It changes no action availability, and no service caller becomes executable before AUTH-09E. - Current ART gate: integrate trusted `main`, complete deterministic 02C1 - proof, and pass all nine exact-SHA internal reviewer tracks before opening - the ART PR. No later ART chunk starts automatically. + proof, and pass all nine exact-SHA internal reviewer tracks. Those steps are + complete; publish the final candidate to existing PR #154, then pass fresh + external checks and explicit human review. No later ART chunk starts + automatically. - Scope checkpoint: AWS S3 is the only v0.1 production provider; MinIO is local/CI S3 protocol proof; LocalStorage is focused development/test; R2 and Flow Node are deferred. Product modules receive narrow artifact capabilities, diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md index fec9fb02f..07efb6332 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md @@ -15,8 +15,10 @@ The planning-only cross-initiative boundary reconciliation merged through PR #139 as `5d353b6`, and AUTH's owner reconciliation merged through PR #140 as `d541521`. ART now consumes AUTH's canonical activation-custody and prepared mutation contracts without editing or activating AUTH runtime behavior. -AUTH-09D-A merged through PR #148 as `99ae4c9` and is integrated into the ART -candidate; AUTH-09D-B remains inactive. +AUTH-09D-A merged through PR #148 as `99ae4c9`, AUTH-09D-B merged through PR +#152 as `93dd392`, and the contributor foundation merged through PR #153 as +`8d5eb15b`; all are integrated into the ART candidate. AUTH-09E remains +inactive. The Flow Node-focused amendment candidate `6cc422d` passed deterministic checks but failed internal review on recovery/API completeness. Before repair, the user @@ -44,6 +46,12 @@ scope. Provider execution, verification, publication, recovery, routes, and product cutover remain inactive. Native AWS remains runtime-ineligible. R2 and Flow Node remain deferred. +Final implementation SHA `535069cfb1a7312d731bb14a6023ceb0894402e9` +passed 371 focused tests with 94.02 percent scoped coverage and all nine +required internal reviewer tracks. The current gate is publication of that +reviewed candidate to existing PR #154 followed by fresh GitHub and CodeRabbit +evidence. + ## Next Proposed Chunk `02C2` may add fenced put resolution and verification publication only after @@ -53,8 +61,8 @@ has a v0.1 chunk. ## Gate The current gate is deterministic 02C1 proof followed by all nine exact-SHA -internal reviewer tracks. GitHub Actions, CodeRabbit, and explicit human review -follow only after internal evidence is complete and every reviewer session is -closed. Provider execution, verification publication, and recovery remain in -later owning chunks. No later artifact chunk starts automatically, and only the -user may approve merge. +internal reviewer tracks; that gate is complete. GitHub Actions, CodeRabbit, +and explicit human review remain pending on the published final candidate. +Provider execution, verification publication, and recovery remain in later +owning chunks. No later artifact chunk starts automatically, and only the user +may approve merge. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md index 3c649f685..08a828f38 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md @@ -2,53 +2,39 @@ ## Boundary -This file records GitHub Actions and CodeRabbit review separately from the -internal sub-agent evidence. It does not replace or modify the internal review -verdicts. +This file records GitHub Actions and CodeRabbit separately from internal +review. It does not replace internal exact-SHA evidence. -Reviewed code SHA: `2595f0ce0964624e8ae022d6cbee04d260826612` +Reviewed implementation SHA: `535069cfb1a7312d731bb14a6023ceb0894402e9` -Current evidence-only head before this record: `7ca05d5b4d54bcc337d107de781e6e289135c2d6` +Trusted base: `8d5eb15b384fd75787ce98a099400a1d335d2560` -## GitHub Actions +PR: #154, `https://github.com/Flow-Research/workstream/pull/154` -The first Agent Gates run failed closed because the already-declared `02C2` -contract heading omitted the canonical ` - ` separator required by merge-intent -validation. The merge intent was not weakened or changed to a null successor. -The contract heading was corrected, `02C2` remained proposed and inactive, and -local merge-intent validation plus all 88 Agent Gates passed. +## Historical Evidence -The replacement Agent Gates run passed. Backend is rerunning the authoritative -isolated full suite and 78 percent repository floor on the final external-review -evidence head. - -## CodeRabbit - -CodeRabbit completed with no review submission, no inline review thread, and no -actionable comment. - -Its pre-merge summary raised two warnings: - -1. The generic docstring calculation reported 54.37 percent. No code change was - made for this warning because the repository's authoritative configured - command, `docstr-coverage --config .docstr.yaml`, passed at 90.4 percent - against the enforced 80 percent floor. -2. The original PR description did not reproduce every trust-bundle section. - This was valid metadata feedback. The PR description now includes the chunk - and merge intent, approved intent, design, alternatives, scope, evidence, - acceptance proof, test delta, reviewer table, CI integrity, risks, follow-up, - human focus, and merge-ownership checklist. +Earlier GitHub and CodeRabbit results ran on pre-rebase heads and are not +evidence for the reviewed implementation SHA. The old remote Backend failure +was caused by stale migration-head expectations before the contributor +foundation rebase; local current-SHA migration and focused proof pass. No old +external PASS is carried forward. ## Current Status | Source | Status | Notes | |---|---:|---| -| CodeRabbit | PASS | No actionable review threads; both warnings were assessed and the valid metadata warning was fixed. | -| Agent Gates | PASS | Canonical successor validation and all repository agent gates pass. | -| Backend | Pending | Final evidence-only head must complete the isolated full suite and coverage gates. | +| GitHub Agent Gates | Pending | Await publication of the rebased final candidate. | +| GitHub Backend | Pending | Must run the full isolated suite, all scoped gates, and 78-percent floor on the final head. | +| CodeRabbit | Pending | Request a fresh review on the published final head. | | Human review | Pending | Only the user may approve merge. | +## Response Rule + +Assess only external findings verified against the published final head. Any +implementation, test, workflow, policy, specification, or chunk-contract repair +requires affected internal reviewers to rerun and evidence to be rebound. + ## Stop Condition -Wait for the final GitHub Backend result and explicit human review. Do not +Wait for fresh external checks and explicit user approval of PR #154. Do not merge and do not start `WS-ART-001-02C2` automatically. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md index 9fabc104d..072f7395b 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md @@ -4,102 +4,111 @@ `WS-ART-001-02C1`: Admission And Put-Attempt Foundation -open sub-agent sessions: none +Open sub-agent sessions: none. -valid findings addressed: yes +Valid findings addressed: yes. ## Reviewed Revision -Reviewed code SHA: `2595f0ce0964624e8ae022d6cbee04d260826612` +Reviewed implementation SHA: `535069cfb1a7312d731bb14a6023ceb0894402e9` -Reviewed at: 2026-07-19T08:58:54Z +Trusted base: `8d5eb15b384fd75787ce98a099400a1d335d2560` -Reviewer run IDs: senior-engineering=019f797a-e6b0-7920-8f87-bb01c93fb972; QA/test=019f797a-e989-7441-9d6d-f52ca2a20276; security/auth=019f797a-ed1b-7c63-b8d6-2143a4152d07; product/ops=019f7988-7323-7851-b17d-8452f9db01c0; architecture=019f7988-7bfc-7670-b9bb-67342ee934b3; reuse/dedup=019f797b-03a8-7cd0-a5de-b8e5ae3c052b; CI-integrity=019f7988-8341-7b83-8324-11461058509a; test-delta=019f7988-908c-7873-a536-2360ebb123bf; docs=019f7988-a143-7a53-a3b6-c8a304d06c22 +Reviewed on: 2026-07-19. -The reviewed base is trusted `main` at -`93dd392484b397cfdfaaa833631dc2c27f591ed7`, including merged AUTH PR #152. -Only review artifacts and initiative status may change after the reviewed SHA. -Any implementation, test, workflow, policy, or chunk-contract change invalidates +Reviewer sessions: senior engineering, security/auth, and CI integrity used +`/root/review_senior_6392825f`; architecture, product/ops, and test delta used +`/root/review_arch_6392825f`; QA/test, reuse/dedup, and docs used +`/root/plan_review_actor_boundary`. Each track explicitly rebound its review to +the final SHA above. All sessions completed. + +Only review evidence, trust-bundle, external-response, and initiative-status +files may change after the reviewed implementation SHA. Any implementation, +test, workflow, policy, specification, or chunk-contract change invalidates this evidence and requires a new exact-SHA review cycle. ## Reviewed Change -- Added durable task, producer, project, and deployment admission scopes with - server-owned limits and unique content charges. -- Added closed guide, contributor, and checker-output admission requests whose - relationships and producer authority are resolved and locked by Workstream. -- Atomically claims the storage namespace, reserves capacity, writes durable - admission evidence, and creates one `prepared` `ArtifactPutAttempt` before - provider I/O. -- Deduplicates exact replay while reacquiring released charges only after - capacity and linked-charge revalidation. -- Locks guide source items and snapshots during authoritative admission so the - persisted attempt cannot race mutable guide facts. -- Keeps provider execution, verification, publication, recovery, routes, and - product cutover out of scope and inactive. -- Adds migration `0027` with populated-state downgrade refusal across scopes, - charges, attempts, and attempt-charge links. +- Adds server-owned deployment, project, producer, and task admission scopes, + unique content charges, and one atomic prepared put attempt before provider + I/O. +- Accepts only closed guide, contributor, and checker-output requests and + derives canonical relationships and scope limits inside Workstream. +- Uses an actors-owned same-transaction proof boundary that locks the exact + profile then identity link and returns frozen primitive state; ART no longer + imports or queries actor persistence. +- Leaves every prepared execution field inactive: `next_run_at`, executor, + lease, terminal result, replica, and receipt are null and execution generation + is zero. +- Makes release timestamps biconditional with released charge state. +- Adds deterministic database contention proof for same-content deduplication + and distinct-content oversubscription. +- Adds migration `0028_artifact_admission`, following + `0027_contributor_foundation`, with populated-state downgrade refusal. ## Reviewer Results -| Reviewer | Result | Blocking findings | Notes | +| Reviewer | Result | Blocking findings | Disposition | |---|---:|---|---| -| senior engineering | PASS | None | Transaction, replay, rollback, failure, and scope behavior are maintainable and bounded. | -| QA/test | PASS | None | Real PostgreSQL concurrency, lock, rollback, relationship, and migration cases satisfy the contract. | -| security/auth | PASS | None | Exact actor, identity-link, service-identity, relationship, scope, and replay checks fail closed. | -| product/ops | PASS | None | Admission remains internal and does not alter task, review, revision, contribution, or compensation lifecycle state. | -| architecture | PASS | None | Provider-neutral boundaries and transaction ownership are preserved with no route or provider execution drift. | -| reuse/dedup | PASS | None | Canonical artifact interfaces are reused and no parallel provider-reference or admission abstraction remains. | -| CI integrity | PASS | None | The new audit coverage gate is additive; the repository 78 percent floor and cumulative 90 percent gates remain fail closed. | -| test delta | PASS | None | No skipped or weakened tests; removed direct-write tests correspond to the deliberately removed provider-finalization path. | -| docs | PASS | None | Active docs, migration behavior, terminology, scope exclusions, and links match the implementation. | - -## Valid Findings Addressed - -- Replaced read-only guide admission facts with row locks on both the exact - source item and its immutable snapshot, plus a two-transaction lock-timeout - regression test. -- Proved capacity failure leaves no partial admission state. -- Proved an attempt-only populated state prevents destructive migration - downgrade. -- Moved provider-object reference parsing and construction behind the canonical - provider-neutral artifact interface. -- Revalidated canonical active human profiles and identity links during guide - and contributor admission. -- Revalidated replay capacity and the exact attempt-charge set before returning - an existing attempt. -- Bound checker output to the canonical submission and task relationship. -- Corrected the declared `02C2` successor contract heading to the canonical - merge-intent grammar without starting or changing `02C2` behavior. -- Replaced inaccurate audit-event wording with the exact durable admission - evidence and admission-ledger terms used by this chunk. - -## Commands Run - -```bash -cd backend && WORKSTREAM_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_artifact_admission.py tests/test_artifact_architecture.py tests/test_artifact_cleanup_wiring.py tests/test_artifact_preparation.py tests/test_artifact_store_conformance.py tests/test_artifacts.py tests/test_local_artifact_store.py tests/test_s3_artifact_store.py tests/test_audit.py tests/test_config.py -q --cov=app.interfaces.artifact_operations --cov=app.modules.artifacts --cov=app.modules.audit --cov=app.core.config --cov-report=term-missing --cov-fail-under=90 -cd backend && .venv/bin/ruff check app tests -python3 scripts/check_stale_artifact_contracts.py -python3 scripts/test_agent_gates.py -python3 scripts/check_markdown_links.py -git diff --check +| senior engineering | PASS | None | Transaction and ownership repairs are bounded and maintainable. | +| architecture | PASS | None | Actors own persistence proof; ART remains provider-neutral and dormant. | +| QA/test | PASS | One repaired High | Distinct actors and a scope-reservation barrier now prove real ledger contention. | +| security/auth | PASS | None | Exact identity, relationship, configuration, and quota checks fail closed without duplicating AUTH. | +| product/ops | PASS | None | No task, submission, checker, review, contribution, compensation, or reputation lifecycle mutation. | +| reuse/dedup | PASS | None | Canonical actor and artifact interfaces are reused; optional test-only barrier duplication is documented. | +| CI integrity | PASS | None | All prior 90-percent gates and the repository 78-percent floor remain fail closed. | +| test delta | PASS | None | Removed tests match intentionally removed provider execution; retained behavior gains stronger proof. | +| docs | PASS after regeneration | Evidence was stale | Migration, inactive scheduling, status, evidence, and trust bundle are synchronized. | + +## Findings Addressed + +- Removed ART's direct `ActorProfile` and `ActorIdentityLink` queries and added + an actors-owned frozen admission proof in the caller's transaction. +- Made `next_run_at` nullable and null for `prepared`, enforced by the prepared + execution-inactive database constraint. +- Enforced `(state = 'released') = (released_at is not null)`. +- Preclaimed the namespace in concurrency fixtures, used distinct actors, and + synchronized immediately before the real scope reservation method. The + same-content case proves two attempts, seven unique charges, eight links, and + four counted deployment bytes; oversubscription proves one success, one typed + capacity failure, one attempt, and four charges. +- Corrected active migration wording from `0027` to + `0028_artifact_admission`. + +## Deterministic Proof + +The isolated real-PostgreSQL/MinIO focused matrix ran against the reviewed SHA: + +```text +371 passed in 757.38s +scoped coverage: 94.02% +required scoped floor: 90% +Alembic head: 0028_artifact_admission ``` -Results: 369 tests passed in 650.13 seconds with 94.32 percent scoped -coverage. Ruff, the stale artifact contract scan, 88 agent-gate tests, -Markdown links, and diff integrity passed. GitHub Backend CI remains -authoritative for the isolated full repository suite and 78 percent floor. +Additional results: + +- Ruff: PASS. +- configured docstring coverage: PASS at 90.5 percent. +- stale artifact contract scan: PASS at `artifact_store_cutover`. +- agent gates: PASS, 88 tests. +- Markdown links: PASS. +- schema-v2 merge-intent validation: PASS. +- `git diff --check`: PASS. + +GitHub Backend remains authoritative for the isolated full repository suite and +78-percent repository-wide floor on the published final head. ## Remaining Risks -- Provider execution, provider acknowledgement verification, and publication - remain intentionally unavailable until separately approved later chunks. -- Native AWS remains runtime-ineligible pending its separately owned live proof. -- The admission service remains an internal foundation and is not yet wired to - project or submission product routes. +- Provider execution, acknowledgement observation, verification, publication, + recovery, routes, and product cutover remain intentionally unavailable. +- Native AWS remains runtime-ineligible pending separately owned live proof. +- External GitHub and CodeRabbit checks remain pending until the rebased final + candidate is published. ## Stop Condition -Publish this evidence-bound candidate for GitHub Actions, CodeRabbit, and -explicit human review. Do not merge without the user's approval and do not -start `WS-ART-001-02C2` automatically. +Publish the evidence-bound candidate to existing PR #154, wait for fresh +external checks, and stop for explicit user merge approval. Do not start +`WS-ART-001-02C2` automatically. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md index 5229ffb8e..df39978c7 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-pr-trust-bundle.md @@ -1,100 +1,104 @@ # PR Trust Bundle: WS-ART-001-02C1 -## Chunk +## Chunk And Intent `WS-ART-001-02C1` - Admission And Put-Attempt Foundation -Merge intent: `.agent-loop/merge-intents/WS-ART-001-02C1.json` - -## Goal +Reviewed implementation SHA: `535069cfb1a7312d731bb14a6023ceb0894402e9` -Create the durable admission and prepared-attempt transaction required before -any artifact provider write, while keeping execution and product cutover -inactive. +Trusted base: `8d5eb15b384fd75787ce98a099400a1d335d2560` -## What Changed +Merge intent: `.agent-loop/merge-intents/WS-ART-001-02C1.json` -- Added server-derived task, producer, project, and deployment byte limits. -- Added unique provisional/completed/released content charges with exact replay - deduplication and concurrent oversubscription protection. -- Added closed guide, contributor, and checker-output admission commands. -- Added one atomic namespace-claim, capacity-reservation, durable admission - evidence, and prepared `ArtifactPutAttempt` transaction. -- Added exact human and service identity revalidation, canonical relationship - checks, guide-source row locking, and replay charge-set validation. -- Added migration and real PostgreSQL concurrency, rollback, and downgrade - proofs. +Create the durable PostgreSQL admission and prepared-attempt transaction that +must commit before provider I/O while keeping execution and product cutover +inactive. -## Scope Control +## Design And Scope -This chunk adds no provider write, provider observation, verification, -publication, Celery execution, recovery attempt, Operator/public route, product -cutover, task claim, reviewer lease, R2 path, or Flow Node path. +- Workstream derives deployment, project, producer, and applicable task scopes. +- Unique provisional/completed charges count once per scope and exact content; + released charges may be reacquired only under locked capacity checks. +- Actors owns exact profile/link locking and returns frozen primitive proof in + the caller's admission transaction; ART makes no AUTH permission decision. +- One namespace claim, complete charge set, and `prepared` attempt commit + atomically. Prepared scheduling/execution fields remain inactive and null. +- No provider write/observation, verification, publication, recovery, Celery + execution, route, product cutover, task claim, reviewer lease, R2, or Flow + Node path is included. ## Acceptance Proof -- [x] Callers cannot supply or weaken admission scopes. -- [x] Same content is charged once per applicable scope. -- [x] Concurrent reservations cannot double-charge or oversubscribe. -- [x] Reservation failure leaves no attempt, charge, admission evidence, or provider call. -- [x] One committed prepared attempt is required before later provider work. -- [x] Guide facts, human authority, service identity, and producer relationships are revalidated transactionally. -- [x] Provider execution fields remain inactive and no product route exposes admission. -- [x] Exactly one schema-v2 merge intent names only `02C2`, which still requires an explicit start. +- [x] Callers cannot supply, omit, or weaken admission scopes or limits. +- [x] Exact actor/link and canonical product relationships are locked and + revalidated without crossing actor persistence ownership. +- [x] Same content is charged once per applicable scope under real concurrent + ledger contention. +- [x] Concurrent distinct content cannot oversubscribe a shared scope. +- [x] Reservation failure leaves no partial charge, attempt, receipt, audit + success, or provider call. +- [x] A committed prepared attempt is required before any later provider work. +- [x] Prepared `next_run_at`, executor, lease, terminal, replica, and receipt + fields are null and execution generation is zero. +- [x] Exactly one schema-v2 merge intent names only inactive successor `02C2` + and requires a separate explicit start. ## Tests And Checks ```text -369 focused tests PASS -Scoped changed-subsystem coverage 94.32% +371 focused tests PASS in 757.38s +Scoped changed-subsystem coverage 94.02% (required 90%) +Alembic head 0028_artifact_admission Ruff PASS +Configured docstring coverage PASS at 90.5% Stale artifact contract scan PASS 88 agent-gate tests PASS Markdown links PASS +Schema-v2 merge-intent validation PASS git diff --check PASS ``` -GitHub Backend CI remains authoritative for the isolated full suite and 78 -percent repository coverage floor. +GitHub Backend remains authoritative for the full isolated repository suite and +78-percent repository coverage floor. ## Internal Review -Reviewed code SHA: `2595f0ce0964624e8ae022d6cbee04d260826612` - -All nine required reviewer tracks passed with no unresolved findings. Every -reviewer session is closed. Run IDs and repaired findings are recorded in the -internal review evidence. +All nine required tracks reviewed exact SHA `535069cf...`: senior engineering, +architecture, QA/test, security/auth, product/ops, reuse/dedup, CI integrity, +test delta, and docs. QA's initial High concurrency-proof finding was repaired +and passed on rerun. Docs' stale-evidence finding was repaired by regenerating +the exact-SHA evidence, this bundle, external status, and initiative state. No +blocking finding remains and every reviewer session is closed. ## External Review -External checks are separate from internal agent review. Their detailed status -and response are recorded in -`.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md`. - | Source | Status | Notes | |---|---:|---| -| GitHub Actions | Running | Replacement Agent Gates passed; Backend must complete on the final evidence-only head. | -| CodeRabbit | PASS | No actionable review thread; the valid PR-description warning was fixed and the repository docstring gate independently passes. | -| Human review | Pending | Only the user may approve merge. | +| GitHub Agent Gates | Pending | Must run on the published rebased final head. | +| GitHub Backend | Pending | Must prove the full suite, scoped gates, and 78-percent floor. | +| CodeRabbit | Pending | A fresh current-head review must be requested after publication. | +| Human review | Pending | Only the user may approve PR #154 for merge. | ## Remaining Risks -- Admission is not yet connected to provider execution or product submission - routes; those remain later, separately approved chunks. -- Native AWS remains unavailable until its release-bound live proof. +- Admission is intentionally not connected to provider execution or product + submission routes. +- Native AWS remains unavailable until its separately owned live proof. +- External checks have not yet run on the final rebased head. ## Human Review Focus -- Can a caller omit a scope, bypass a limit, or substitute producer context? -- Does every successful admission create exactly one durable prepared attempt - before any provider side effect? -- Do rollback and exact replay preserve byte accounting and admission-ledger integrity? -- Are provider execution, verification, recovery, and product routes still absent? +- Can a producer omit a scope, weaken capacity, or substitute identity/context? +- Does every success commit the complete ledger and exactly one prepared + attempt before provider side effects? +- Do rollback, replay, release/reacquisition, and concurrent contention preserve + exact accounting? +- Are provider execution, verification, recovery, and product routes absent? ## Human Merge Ownership - [ ] I can explain what changed and why. - [ ] I know what could break. - [ ] I accept the remaining risks. -- [ ] GitHub CI and external review pass. -- [ ] The user explicitly approved this PR for merge. +- [ ] GitHub CI and external review pass on the final head. +- [ ] The user explicitly approved PR #154 for merge. From c93f1a246ddbc595aa0676a63c2926bf26fc1612 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sun, 19 Jul 2026 16:29:31 +0100 Subject: [PATCH 20/20] Fix ART review evidence schema --- ...S-ART-001-02C1-external-review-response.md | 27 +++++++++++++++++++ ...S-ART-001-02C1-internal-review-evidence.md | 17 ++++++------ 2 files changed, 35 insertions(+), 9 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md index 08a828f38..f77dca201 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-external-review-response.md @@ -28,6 +28,33 @@ external PASS is carried forward. | CodeRabbit | Pending | Request a fresh review on the published final head. | | Human review | Pending | Only the user may approve merge. | +## Current-Head Triage + +Comments addressed: + +- The first Agent Gates and Backend runs on evidence head `a93be2ec` failed at + the shared internal-review evidence parser before tests. The regenerated + evidence used non-canonical provenance labels and verdict text. +- Evidence now uses the required `Reviewed code SHA`, UTC `Reviewed at`, and + `Reviewer run IDs` labels. QA and docs use canonical `PASS after fixes` with + no remaining blocking findings. + +Comments deferred: none. + +Human decisions needed: explicit merge approval only after external checks pass. + +Commands rerun: + +```text +PR_HEAD_SHA=a93be2ec25689e1f8e036321d8a45b1fe35455ed python3 scripts/check_internal_review_evidence.py: PASS +python3 scripts/test_agent_gates.py: PASS, 88 tests +python3 scripts/check_markdown_links.py: PASS +git diff --check: PASS +``` + +Remaining risks: GitHub and CodeRabbit must complete against the corrected +published evidence head. + ## Response Rule Assess only external findings verified against the published final head. Any diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md index 072f7395b..b988d5ba1 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-02C1-internal-review-evidence.md @@ -10,17 +10,16 @@ Valid findings addressed: yes. ## Reviewed Revision -Reviewed implementation SHA: `535069cfb1a7312d731bb14a6023ceb0894402e9` +Reviewed code SHA: `535069cfb1a7312d731bb14a6023ceb0894402e9` Trusted base: `8d5eb15b384fd75787ce98a099400a1d335d2560` -Reviewed on: 2026-07-19. +Reviewed at: 2026-07-19T15:20:00Z -Reviewer sessions: senior engineering, security/auth, and CI integrity used -`/root/review_senior_6392825f`; architecture, product/ops, and test delta used -`/root/review_arch_6392825f`; QA/test, reuse/dedup, and docs used -`/root/plan_review_actor_boundary`. Each track explicitly rebound its review to -the final SHA above. All sessions completed. +Reviewer run IDs: senior-engineering/security-auth/ci-integrity=/root/review_senior_6392825f; architecture/product-ops/test-delta=/root/review_arch_6392825f; qa-test/reuse-dedup/docs=/root/plan_review_actor_boundary + +Each track explicitly rebound its review to the final SHA above. All sessions +completed. Only review evidence, trust-bundle, external-response, and initiative-status files may change after the reviewed implementation SHA. Any implementation, @@ -52,13 +51,13 @@ this evidence and requires a new exact-SHA review cycle. |---|---:|---|---| | senior engineering | PASS | None | Transaction and ownership repairs are bounded and maintainable. | | architecture | PASS | None | Actors own persistence proof; ART remains provider-neutral and dormant. | -| QA/test | PASS | One repaired High | Distinct actors and a scope-reservation barrier now prove real ledger contention. | +| QA/test | PASS after fixes | None | Distinct actors and a scope-reservation barrier now prove real ledger contention. | | security/auth | PASS | None | Exact identity, relationship, configuration, and quota checks fail closed without duplicating AUTH. | | product/ops | PASS | None | No task, submission, checker, review, contribution, compensation, or reputation lifecycle mutation. | | reuse/dedup | PASS | None | Canonical actor and artifact interfaces are reused; optional test-only barrier duplication is documented. | | CI integrity | PASS | None | All prior 90-percent gates and the repository 78-percent floor remain fail closed. | | test delta | PASS | None | Removed tests match intentionally removed provider execution; retained behavior gains stronger proof. | -| docs | PASS after regeneration | Evidence was stale | Migration, inactive scheduling, status, evidence, and trust bundle are synchronized. | +| docs | PASS after fixes | None | Migration, inactive scheduling, status, evidence, and trust bundle are synchronized. | ## Findings Addressed