Skip to content

feat: add run quality assurance - #100

Merged
pcvantol merged 94 commits into
mainfrom
codex/ep-run-quality-assurance
Sep 8, 2026
Merged

feat: add run quality assurance#100
pcvantol merged 94 commits into
mainfrom
codex/ep-run-quality-assurance

Conversation

@pcvantol

@pcvantol pcvantol commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Doel

Implementeert EP_RUN_QUALITY_ASSURANCE_V1: onafhankelijke Quality- en
Security-review, durable assurance-evidence, begrensde herstelrondes en
bewijspariteit in lifecycle, receipt en dashboard.

Kernwijzigingen

  • Quality en Security zijn afzonderlijke, technisch read-only reviews van
    dezelfde kandidaat. Nieuwe Managed PR-publicatie blijft achter actuele lokale
    validatie en beide vereiste reviews.
  • Review-, kandidaat-, profiel- en herstelbewijs blijft append-only in CENTRAL;
    de dashboardmodal toont dezelfde quality/security- en herstelidentiteiten als
    de historische projectie.
  • De Operations Console heeft per-project queue-dispositions met gelokaliseerde
    bevestigingsmodals en toestandafhankelijke, rechts uitgelijnde actieknoppen.
  • De queue-statemachine is expliciet: QUEUED → DEFERRED|QUARANTINED|DECLINED,
    DEFERRED → QUEUED, QUARANTINED → QUEUED|DECLINED; DECLINED is
    terminaal. Directe quarantaine-afwijzing voorkomt een worker-eligible window.
  • Queue-mutaties zijn CENTRAL-transacties met operation-ID-idempotency,
    expected-state/revision-CAS, append-only eventbewijs en claim-raceprotectie.
    Een geclaimde submission geeft 409; een actie annuleert nooit een actieve
    run.
  • Producer-readback v1.2 projecteert de disposition apart van het
    uitvoeringsresultaat. Een onuitgevoerde afwijzing krijgt dus geen gefingeerde
    run, commit of terminal receipt.

Autorisatie en contracten

  • Origincontrole is uitsluitend CSRF-bescherming. De server eist daarnaast een
    actieve project-scoped credential en een expliciete capability:
    QUEUE_HOLD_RESUME of QUEUE_DECLINE.
  • Capability-grants zijn beheerbaar via de EP Server-CLI en worden duurzaam
    bijgehouden. Producercredentials verlenen geen operatorrechten.
  • Queue HTTP JSON v1.0 weigert duplicaatvelden, onbekende/missende velden,
    verkeerde types, stale modals en conflicterend operation-ID-hergebruik.
  • OpenAPI/Postman en de producer-readbackdocumentatie zijn bijgewerkt; de
    kwalificatie test 401 zonder bearer, 403 zonder capability en de
    geautoriseerde CAS-flow inclusief QUARANTINED → DECLINED.

Migratie en herstel

  • Schema 56 introduceert de capability-opslag als afzonderlijke evolutie.
  • De oudere schema-54/55-herstelroute gebruikt expliciete kolommapping en is
    idempotent voor reeds aanwezige additieve queue-evidence. Dit herstelt een
    aangetroffen upgradefout zonder handmatige databasepatch of tweede authority.

Kwalificatie en actuele status

  • Installed P-TRANSPORT 3×2, deterministische Genesis/Managed E2E,
    HTTP/OpenAPI/Postman en alle vier dashboardshards zijn op recente heads groen
    uitgevoerd.
  • Trusted Delivery, CodeQL, security baseline, versioning, golden en
    lokalisatiechecks zijn groen op recente heads.
  • De per-module coveragegrens blijft 80.20%; de laatste meting faalde nog
    uitsluitend op execution_host.py en server.py. Gerichte regressietests
    voor de nieuwe server- en providerprocessgrenzen zijn toegevoegd en een
    nieuwe CI-meting draait. Deze PR claimt daarom nog geen volledig groene head.

Risico en grenzen

  • De deterministische provider is uitsluitend een geïsoleerde
    kwalificatiecompositie; normale productcompositie blijft fail-closed.
  • De optionele echte GitHub-writevariant vereist expliciete opt-in, een exact
    geautoriseerde disposable repository en gecontroleerde cleanup; zij is geen
    standaard-CI-remote-write.
  • Geen dashboardfallbacks, productie-exclusions, verlaagde coveragegrenzen of
    gefabriceerde execution-evidence toegevoegd.

SemVer-reconciliatie

De product-eigen SemVer-grens is aangescherpt op de actuele PR-head: de TOML-writer is begrensd tot [project].version; beide eigen package-lock.json rootvelden worden gevalideerd en geprojecteerd; alle projecties worden vóór writes gevalideerd. De gewone wheelbuilder blijft read-only.

Bron- en installed-consistency zijn nu afzonderlijk: de installed-check draait via een geïsoleerde interpreter tegen het daadwerkelijk geïnstalleerde wheel en weigert source-path shadowing. EP #105 blijft de afzonderlijke, nog niet geïnstalleerde source-level delivery-adapter; deze PR claimt geen grant, protected merge, publicatie of runtime-installatie.

Bootstrap release cadence V2

BOOTSTRAP_RELEASE_CADENCE_V2: EP helper en operationele documentatie zijn read-only-CI reconciled; docs-only/no-allocation is regressiegedekt. De nog onafhankelijke assurance- en Owner Authorization-gates blijven ongewijzigd.

@pcvantol
pcvantol force-pushed the codex/ep-run-quality-assurance branch from 99870f5 to 2ce2d55 Compare September 7, 2026 19:10
@pcvantol
pcvantol force-pushed the codex/ep-run-quality-assurance branch from 75a6a71 to f60818b Compare September 7, 2026 20:18

pcvantol commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Bounded SemVer architecture handoff — version-preparation delivery owner

Shared clarification for the owner-reported blocker: pcvantol/forge#49 (comment) . This does not close the independent QA R1–R5, queue QD1–QD7 or existing SemVer implementation findings and grants no merge/install/publish permission.

The product repositories own version sources/policy/helpers. EP owns the bounded version-operation-to-candidate execution, using its existing Managed Git/GitHub delivery boundary and the actually configured scoped actor. Inspected EP main fea81e81 already exposes GitProvider/GitHubProvider (gh) and GhCliClient; this is the provider seam to extend/reuse, not proof of a qualified version-specific adapter or installed credentials. Forge/Workspace do not need separate privileged GitHub bots, and the generic Forge releaseplanner is not required.

Implement the minimal operation adapter under existing admission/lease/qualification/evidence semantics: verify operation authorization, expected baseline/head, source-event provenance, policy digest and allowlisted paths; apply the product helper; publish manifest+prepared receipt as one candidate; bind exact head/PR/checks/merge separately through genuine delivery evidence. A checked receipt is not a grant. Avoid a self-referential commit loop by never amending that receipt with its own containing commit SHA or subsequent CI result.

Retain per-unique-eligible-source-event allocation and one feature bump per lineage; no extra bump on retries, check runs, repair SHA or the version-operation's own candidate/finalization events. Batching may combine candidate publication but must retain and count its frozen event set. Enforce existing independent assurance/review and repository merge policy. No arbitrary command runner, new execution mode, queue, event daemon, PAT, App provisioning or protection bypass is requested.

Establish source/controlled candidate qualification via existing authorized bootstrap development if necessary; do not circularly demand the new adapter be installed before its own PR can be written. Autonomous event-to-EP routing and installed writer qualification must remain explicitly unproven until tested. Coordinate ownership with the active assurance/queue and SemVer sessions; no concurrent writers on #100. No source, runtime, grant, budget or permission changes were made by this comment.

@pcvantol

pcvantol commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Coordination: the generic EP version-preparation execution seam now has its own implementation PR #105 (031f8ef) from current main, per the delivery-ownership clarification. #100 remains untouched and should reconcile its EP-specific multi-projection helper only after #105 is reviewed/merged. #105 has no credential, grant, runtime or direct-main mutation and does not claim installed writer qualification.

pcvantol commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Architect follow-up — exact-head hosted outcome after diagnostic change

Reviewed head: e8754a0113e7543fb1c3decfcfd2f2448c0ea6c9. Read-only review of the completed GitHub run/logs; no source, runtime, credentials, grant, check status or merge changes.

The previously RUNNING validation is now FAILURE, run 34255514602, job 102160221758: https://github.com/pcvantol/engineering-platform/actions/runs/34255514602/job/102160221758 . The checked-out test candidate was GitHub's synthetic merge 649e7bf5b144df9f43f968b5591efe773f5c88dd (head e8754a0 into base 7b00321), not a new delivery commit.

Observed on that run:

  • committed-version wheel build/install: PASS, version 2.3.0;
  • source/installed version check: EP_VERSION_COMPONENTS=PASS version=2.3.0 components=15;
  • unit suite: 1,311 tests, OK;
  • all four browser-dashboard shards, including screenshot upload: PASS;
  • installed P-TRANSPORT 3x2: FAILURE;
  • subsequent deterministic Genesis/Managed E2E, HTTP/OpenAPI/Postman and coverage steps: SKIPPED, not qualified on this run;
  • separate Owner Authorization commit status remains failure. A successful Trusted Delivery workflow envelope is not that status passing.

The new bounded diagnostics did their job. The failing operation is:

bootstrap-topology --data-root <temporary qualification root>/central
  --project-id durability-accept
  --repository-id durability-accept-repo

The CLI reports:

{"error":"EP Server store is unavailable.","ready":false}

The preceding server output contains several BrokenPipeError exceptions while replying from the do_GET -> _send path. This narrows the failure to the installed fixture's store/CLI/HTTP activity; it does not establish whether the root cause is transaction contention, migration/reinitialization, fixture ordering, response latency or another storage error. Do not declare a timing-only flake or dismiss it merely because a local run passed.

Next bounded action: inspect the real storage exception behind the CLI's generic store-unavailable result, retain safe redacted exception/category/timing details in qualification diagnostics, reproduce the actual phase transition around durability-accept with the installed artifact and simultaneous server/worker activity, and add the regression at the failing boundary. Inspect lock/transaction lifetime and repeated schema initialization as hypotheses, not as a proven diagnosis. Do not just suppress BrokenPipe, add unlimited retries, lower thresholds, or reset CENTRAL.

Then run the complete hosted chain again and attach the new exact head/base and coverage evidence. Historical 1,309-test coverage is not the skipped current measurement. This comment adds no new product scope and makes no claim to have re-reviewed all R/QD/SemVer closures.

@pcvantol

pcvantol commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Exact-head qualification update

Reviewed candidate: a06b009b1c9c1ab7177be0c4067df5cf5a665c81, based on main ad5f990e0d862a7a473deaf148f058b66430cb99.

  • Hosted checks for this exact head are green: canonical versioning, CodeQL, AI-development projection, security baseline, Golden Smoke, Trusted Delivery, validation profile, validate, all four browser-dashboard shards, localisation and trusted-delivery evidence. Golden Regression is intentionally skipped.
  • The hosted validation coverage gate passed. Notable previous blockers are now above the unchanged 80.20% floor: execution_host.py 80.34% and server.py 80.56%.
  • Local full regression: 1,339 tests, OK. An isolated temporary-venv wheel installation built and installed engineering-platform 2.3.0, then passed the 15-component version-projection check. The initial system-Python install refusal was correctly due to PEP 668 and was not bypassed.
  • The branch contains current main and has no tracked source drift; the only worktree residue is pre-existing untracked .engineering, untouched.

Remaining exact blocker: the repository status Owner Authorization is FAILURE with HIGH_RISK authorization required for this exact SHA. This is an authorization control, not a code or CI defect. No bypass, fabricated status, credential/grant, runtime or production-publication change was made. The PR is technically qualified pending that authorized decision.

@pcvantol
pcvantol merged commit f7c0887 into main Sep 8, 2026
18 checks passed
@pcvantol
pcvantol deleted the codex/ep-run-quality-assurance branch September 8, 2026 19:24
@pcvantol

pcvantol commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Post-merge exact-SHA closure

PR #100 is normaal gesquasht naar main als f7c08872a2d334cff097ea5f28822836e59f78c3. De eigen push-kwalificatie van die uiteindelijke merge-SHA is volledig geslaagd:

  • Engineering Platform validation: wheel/installatie, P-TRANSPORT 3×2, deterministische Genesis/Managed E2E, HTTP/OpenAPI/Postman, coverage en alle vier browser-shards;
  • CodeQL, security baseline, Canonical versioning, Golden Qualification en TDE observe;
  • de eerder vereiste Owner Authorization gold voor de geverifieerde pre-merge kandidaat a06b009.

Dit is repository- en artifactkwalificatie; geen productiepublicatie, runtime-installatie, credential/grantmutatie of aanvullende release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant