fix(artifacts): bound evaluation content before durable admission by Abiorh001 · Pull Request #485 · Flow-Research/workstream

Abiorh001 · 2026-10-07T13:46:08Z

Change

ARCH-04E1B-B5 — reject unrepresentable evaluation input before durable admission.

A ZIP can fit ART's container limits but exceed the locked checker request limits. Preparation now rejects that input before reserving a pre-check attempt or uploading durable bytes, so successful preparation cannot strand an oversized evaluation input.

Intent and design

Bounded change record · First contributor milestone, step 2

Share CHECKERS' immutable content validation between preparation and later identified requests; retain catalogue identity, field limits, canonical digests and the full 1 MiB envelope.
Use exact locked TASK/PROJECTS facts, actual criteria and every inspected FILE. Project required evidence only from matching verified files.
Reuse existing policy-stamp and evidence-path rules. Move the request helper to the public CHECKERS contract and update affected callers without an alias.
Release prepared scratch before its runtime manager closes, including early rejection.

Preparation does not execute post-submit evaluation. Initial Submission/dispatch is next and must verify stored Submission lineage before reusing the projection. No new route, authority activation, migration, compatibility path or catalogue-limit change.

Scope and test delta

One cohesive ART/TASK/CHECKERS boundary with affected caller/fixture updates and current documentation reconciliation. The file count includes required public-contract consumers and initiative navigation. No unrelated owner implementation or CI gate changes.

New real PostgreSQL/ZIP tests exercise excessive count/path, same-input retry, no new attempt/put/admission, scratch release and a verified positive control. Projection proof covers exact files/evidence, nullable criteria, independent locked-policy substitutions, foreign context and both review modes as detached values. The prior aggregate test now protects the shared content ceiling plus the maximum request envelope, including UTF-8 and one-byte overflow. No tests were skipped or deleted.

Evidence and review

Final reviewed head: 72a973557dc6beea4a9353d214f6e634c94891d4, reconciled with merged CLI-08 at c0c4fe70. The INDEX retains both CLI-08 and ARCH B5; current README/roadmap claims match both boundaries.

Hosted Backend verification: all nine lanes and aggregate passed. All required Agent Gates, authorization, CLI and MCP/API checks passed.
Independent artifact and canonical-validator audit: 8,696 unique tests completed exactly once; zero skips/deselections, valid hashes and complete isolated database/MinIO cleanup. GitHub merge tree 205c2eaa equals the reviewed branch tree. Slowest test lane: 1,136.668 seconds under the unchanged 1,200-second cap.
Focused local proof includes both real PostgreSQL capacity tests, 10 recovery cases and 11 affected PostgreSQL admission/policy cases. Guard-removal probes fail at the intended assertions for shared byte capacity, preparation rejection and TASK body-hash equality. Hosted exact-tree results supersede pre-commit local metadata limitations.
Internal architecture, security, reuse, QA, test-delta, docs/product-operations and CI-integrity reviews pass for the final scoped target. The scratch cleanup-order defect, stale documentation and missed recovery/context fixtures were fixed and re-reviewed. No tests were skipped or deleted and no gate was weakened.
Ruff, module/AUTH boundaries, test structure, behavior ownership, markdown links, stale wording and Commitrail checks passed on compatible scoped targets; reconciliation documentation checks also passed.
CodeRabbit substantively reviewed 17468618 with no actionable comments. Its latest reconciliation review was rate-limited, not substantive; the backend is unchanged between those heads. No unresolved review threads.

Remaining risks and human focus

Review early no-effect rejection, scratch lifetime, exact policy and file projection, and the shared content budget. This remains hidden preparation; it does not deliver the complete contributor journey. Step 3 is atomic initial Submission/ART binding/AUTH receipt/evaluation reservation/outbox dispatch with fresh-authorized replay.

Human approval and merge remain required. No approval is inferred from CI.

coderabbitai · 2026-10-07T13:46:19Z

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 6059b6ba-2be9-47b5-91c0-5bef8374ceee

📥 Commits

Reviewing files that changed from the base of the PR and between 1746861 and e914c73.

📒 Files selected for processing (51)

.commitrail/INDEX.md
.commitrail/initiatives/WS-ARCH-001/OVERVIEW.md
.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1BB5.md
.commitrail/initiatives/WS-ARCH-001/planning/CHUNK_MAP.md
.commitrail/initiatives/WS-ARCH-001/planning/PLAN.md
.commitrail/initiatives/WS-ARCH-001/planning/chunks/WS-ARCH-001-04E-canonical-allow-review.md
.commitrail/initiatives/WS-AUTH-001/OVERVIEW.md
.commitrail/initiatives/WS-CON-001/OVERVIEW.md
.commitrail/initiatives/WS-POL-003/OVERVIEW.md
.commitrail/initiatives/WS-REV-001/OVERVIEW.md
README.md
backend/app/adapters/artifacts/__init__.py
backend/app/adapters/checkers/__init__.py
backend/app/adapters/tasks/__init__.py
backend/app/modules/artifacts/submission_admission.py
backend/app/modules/artifacts/submission_bindings.py
backend/app/modules/artifacts/submission_manifest.py
backend/app/modules/checkers/api/__init__.py
backend/app/modules/checkers/api/artifact_paths.py
backend/app/modules/checkers/api/post_submit.py
backend/app/modules/checkers/api/post_submit_catalogue.py
backend/app/modules/checkers/compiler.py
backend/app/modules/checkers/post_submit_contracts.py
backend/app/modules/tasks/api/submission_context.py
backend/app/modules/tasks/repository.py
backend/app/modules/tasks/service.py
backend/scripts/test_lane_catalogue.py
backend/tests/authorization/task_authority/test_submission_authority.py
backend/tests/authorization/task_authority/test_submission_policy.py
backend/tests/checker_output_admission_helpers.py
backend/tests/checkers/execution/storage_fixture.py
backend/tests/checkers/post_submit/support.py
backend/tests/checkers/post_submit/test_input_bounds.py
backend/tests/checkers/post_submit/test_request.py
backend/tests/post_submit_materialization_helpers.py
backend/tests/retained_material_fixtures.py
backend/tests/submission_capacity_fixtures.py
backend/tests/submission_context_fixtures.py
backend/tests/test_approved_guide_intake.py
backend/tests/test_artifact_bindings.py
backend/tests/test_ci_lane_catalogue.py
backend/tests/test_effective_pre_submit_execution.py
backend/tests/test_pre_submit_attempt_lock_order.py
backend/tests/test_submission_bundle_preparation_recovery.py
backend/tests/test_submission_composition.py
backend/tests/test_submission_evaluation_capacity.py
backend/tests/test_tasks.py
docs/architecture_checker_framework.md
docs/roadmap_status.md
docs/spec_artifact_storage_service.md
docs/spec_chunk_4_task_queue_assignment.md

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 82e44517-f9ed-4144-9664-715122699715

📥 Commits

Reviewing files that changed from the base of the PR and between b6e62f0 and 1746861.

📒 Files selected for processing (51)

.commitrail/INDEX.md
.commitrail/initiatives/WS-ARCH-001/OVERVIEW.md
.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1BB5.md
.commitrail/initiatives/WS-ARCH-001/planning/CHUNK_MAP.md
.commitrail/initiatives/WS-ARCH-001/planning/PLAN.md
.commitrail/initiatives/WS-ARCH-001/planning/chunks/WS-ARCH-001-04E-canonical-allow-review.md
.commitrail/initiatives/WS-AUTH-001/OVERVIEW.md
.commitrail/initiatives/WS-CON-001/OVERVIEW.md
.commitrail/initiatives/WS-POL-003/OVERVIEW.md
.commitrail/initiatives/WS-REV-001/OVERVIEW.md
README.md
backend/app/adapters/artifacts/__init__.py
backend/app/adapters/checkers/__init__.py
backend/app/adapters/tasks/__init__.py
backend/app/modules/artifacts/submission_admission.py
backend/app/modules/artifacts/submission_bindings.py
backend/app/modules/artifacts/submission_manifest.py
backend/app/modules/checkers/api/__init__.py
backend/app/modules/checkers/api/artifact_paths.py
backend/app/modules/checkers/api/post_submit.py
backend/app/modules/checkers/api/post_submit_catalogue.py
backend/app/modules/checkers/compiler.py
backend/app/modules/checkers/post_submit_contracts.py
backend/app/modules/tasks/api/submission_context.py
backend/app/modules/tasks/repository.py
backend/app/modules/tasks/service.py
backend/scripts/test_lane_catalogue.py
backend/tests/authorization/task_authority/test_submission_authority.py
backend/tests/authorization/task_authority/test_submission_policy.py
backend/tests/checker_output_admission_helpers.py
backend/tests/checkers/execution/storage_fixture.py
backend/tests/checkers/post_submit/support.py
backend/tests/checkers/post_submit/test_input_bounds.py
backend/tests/checkers/post_submit/test_request.py
backend/tests/post_submit_materialization_helpers.py
backend/tests/retained_material_fixtures.py
backend/tests/submission_capacity_fixtures.py
backend/tests/submission_context_fixtures.py
backend/tests/test_approved_guide_intake.py
backend/tests/test_artifact_bindings.py
backend/tests/test_ci_lane_catalogue.py
backend/tests/test_effective_pre_submit_execution.py
backend/tests/test_pre_submit_attempt_lock_order.py
backend/tests/test_submission_bundle_preparation_recovery.py
backend/tests/test_submission_composition.py
backend/tests/test_submission_evaluation_capacity.py
backend/tests/test_tasks.py
docs/architecture_checker_framework.md
docs/roadmap_status.md
docs/spec_artifact_storage_service.md
docs/spec_chunk_4_task_queue_assignment.md

💤 Files with no reviewable changes (1)

backend/app/modules/checkers/post_submit_contracts.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

Walkthrough

The change adds bounded post-submission evaluation content and integrates its validation into submission preparation. TASK supplies validated policy lineage and acceptance criteria. ART rejects invalid content before reservation or durable admission. Tests and project records are updated to reflect the new boundary.

Changes

Submission evaluation content

Layer / File(s)	Summary
Content contract and locked policy validation `backend/app/modules/checkers/api/`, `backend/app/modules/checkers/compiler.py`, `backend/app/modules/tasks/...`, `backend/app/adapters/checkers/__init__.py`, `backend/app/adapters/tasks/__init__.py`, `backend/tests/checkers/post_submit/`, `backend/tests/authorization/task_authority/*`, `backend/tests/test_tasks.py`, `backend/tests/submission_context_fixtures.py`, `docs/architecture_checker_framework.md`, `docs/spec_chunk_4_task_queue_assignment.md`, `.commitrail/initiatives/WS-ARCH-001/WS-ARCH-001-04E1BB5.md`	`PostSubmissionEvaluationContent` enforces a canonical content limit of 1 MiB minus 1,024 bytes and checks catalogue and policy consistency. TASK submission facts now include policy lineage and nullable acceptance criteria, with validation against locked project facts and the retained policy body. The request factory and required-evidence path helper are exposed through CHECKERS’ API.
Preparation capacity gate `backend/app/modules/artifacts/...`, `backend/app/adapters/artifacts/__init__.py`, `backend/tests/test_submission_evaluation_capacity.py`, `backend/tests/submission_capacity_fixtures.py`, `backend/tests/test_submission_bundle_preparation_recovery.py`, `backend/tests/test_approved_guide_intake.py`, `backend/tests/test_pre_submit_attempt_lock_order.py`, `backend/scripts/test_lane_catalogue.py`, `backend/tests/test_ci_lane_catalogue.py`, `docs/spec_artifact_storage_service.md`	ART passes locked context, packet facts, manifest file facts, and the archive hash to the evaluation-content factory. Invalid content is rejected before evidence reservation and durable admission. `SubmissionManifest.file_facts()` returns normalized file paths, hashes, and byte counts. Prepared-artifact cleanup is registered with `AsyncExitStack`.
Milestone and next-step records `.commitrail/INDEX.md`, `.commitrail/initiatives/*`, `README.md`, `docs/roadmap_status.md`	Initiative and status records describe bounded evaluation content as delivered before durable admission. They identify atomic initial Submission and dispatch as the next boundary and require it to reuse the bounded-content contract.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant ART preparation
  participant TASK and PROJECTS
  participant CHECKERS
  participant Evidence reservation
  participant Durable put
  ART preparation->>TASK and PROJECTS: obtain locked submission and policy facts
  ART preparation->>CHECKERS: project packet, file facts, archive hash, and locked facts
  CHECKERS-->>ART preparation: bounded content or validation error
  ART preparation->>Evidence reservation: reserve evidence after validation
  ART preparation->>Durable put: admit prepared artifact

Merge Risk: ⚪ Minimal · up to 17468

The change rejects oversized evaluation input before any attempt or durable upload is created. No concrete merge-blocking issue was identified. Wait for the hosted test results to finish before merging.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 58.62% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 58 functions across 35 files. (15 skipped…	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	⚠️ Warning	The description provides strong context for the change, intent, design, scope, tests, evidence, risks, and human review. It does not follow the required template fully and omits or combines required s…	Update the description to use the required template headings. Add the missing sections and provide explicit evidence for commands, results, acceptance criteria, reviewer outcomes, CI and gate integrity, follow-up work, and human merge owner…

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the main change: bounding evaluation content before durable admission.

Full details: Docstring Coverage

Explanation

Docstring coverage is 58.62% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 58 functions across 35 files. (15 skipped: 15 unsupported.)

Full details: Description check

Explanation

The description provides strong context for the change, intent, design, scope, tests, evidence, risks, and human review. It does not follow the required template fully and omits or combines required sections such as Goal, What Changed, Why it Changed, Scope Control, Product Behavior, Commands Run, Result Summary, Acceptance Criteria Proof, Impact-Routed Reviewer Results, External Review, CI And Gate Integrity, Follow-Up Work, and the Human Review Ownership checklist.

Resolution

Update the description to use the required template headings. Add the missing sections and provide explicit evidence for commands, results, acceptance criteria, reviewer outcomes, CI and gate integrity, follow-up work, and human merge ownership.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

Commit to this branch
Create a new PR

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

…ity-alignment # Conflicts: # .commitrail/INDEX.md

Commitrail Probe added 8 commits October 7, 2026 14:04

Plan bounded evaluation capacity before durable admission

e0d99a6

Specify nullable criteria representation at capacity boundary

4128c6f

Validate shared evaluation content before durable submission admission

3f14e16

Keep current capacity navigation aligned with the shared contract

2fc131f

Reconcile remaining capacity references and the next dispatch boundary

05fa1d8

Release prepared scratch before its runtime manager closes

63230e7

Compare early rejection against retained guide-upload custody

d1f8b7e

Assert the approved policy's actual evidence key

bac1182

Abiorh001 marked this pull request as ready for review October 7, 2026 13:50

Commitrail Probe added 2 commits October 7, 2026 15:14

fix(tests): reconcile admission recovery with complete context

1746861

Merge remote-tracking branch 'origin/main' into codex/arch04e1b-capac…

72a9735

…ity-alignment # Conflicts: # .commitrail/INDEX.md

This was referenced Oct 7, 2026

refactor(tasks): remove obsolete payment policy storage and responses #487

Merged

[PILOT-03] Create Submission and initial checker dispatch atomically #490

Closed

Merge branch 'main' into codex/arch04e1b-capacity-alignment

e914c73

abiorh-claw self-requested a review October 8, 2026 07:22

abiorh-claw approved these changes Oct 8, 2026

View reviewed changes

abiorh-claw merged commit 7c8a119 into main Oct 8, 2026
17 checks passed

abiorh-claw deleted the codex/arch04e1b-capacity-alignment branch October 8, 2026 07:23

Abiorh001 mentioned this pull request Oct 8, 2026

feat(cli): declare draft guides and document upload targets #486

Merged

3 tasks done

Navigation Menu

Repository navigation

fix(artifacts): bound evaluation content before durable admission - #485

fix(artifacts): bound evaluation content before durable admission#485
abiorh-claw merged 11 commits into
mainfrom
codex/arch04e1b-capacity-alignment

Abiorh001 commented Oct 7, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 7, 2026 •

edited

Loading

Review limit reached

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (2 warnings)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Abiorh001 commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change

Intent and design

Scope and test delta

Evidence and review

Remaining risks and human focus

Uh oh!

coderabbitai Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (2 warnings)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Abiorh001 commented Oct 7, 2026 •

edited

Loading

coderabbitai Bot commented Oct 7, 2026 •

edited

Loading