Skip to content

Carry complete Goal sources through typed coordination and reconcile cutover work - #5006

Merged
huangruiteng merged 4 commits into
mainfrom
codex/source-projection-transport-0924
Sep 24, 2026
Merged

huangruiteng merged 4 commits into
mainfrom
codex/source-projection-transport-0924

Conversation

@huangruiteng

@huangruiteng huangruiteng commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Complete-source projection on current main fails the existing large File/SQLite fixture before provider admission: the source does not fit the 2 MiB RPC envelope. This PR carries complete sources through the existing typed projection, bootstrap, capture, inspection, qualification and reviewed-promotion handlers using one private, hash-bound local file exchange. A real CLI regression now bootstraps, captures a write, qualifies, promotes and reads back every large record.

The generic RPC budget stays 2 MiB; request/result artifacts have an explicit 16 MiB cap. Wire identities and the byte cap are generated from the existing coordination contract, and Python IO stays in its existing projection adapter. Method, byte length, digest, regular-file identity and private directory checks precede domain execution. Existing TS schemas, source locks, admission and durable receipt owners remain authoritative, and inline callers remain compatible. No provider default or event-writer binding is enabled. Result-delivery failure still requires existing operation-receipt recovery; it never automatically retries a mutation. A related socket-error repair prevents a client that abandons an oversized response from taking down the shared runtime.

The bilingual RFC update withdraws stale “5–8 / 6–8 / 7–9 PRs remaining” claims. It separates merged implementations, six relevant open PRs, four proposed new delivery boundaries and D1–D3 evidence work. The four boundaries are a development plan, not a guaranteed total PR count. See the reconciled frontier. Related: #4574, #4224, #5003; this does not duplicate the event-completion transaction.

Validation:

  • 139 Python tests (118 source/capture/runtime/File/SQLite/real CLI journeys and 21 contract tests); 50 TS projection/transfer/handler/contract tests. Generated-contract and semantic-vocabulary checks pass at the unchanged 43-twin budget.
  • 293 integration tests on a disposable real PostgreSQL 16 server, no skips.
  • Witness-verified detached real-source rehearsal: bootstrap/inspect, complete File/SQLite readback, and legacy/File/PostgreSQL archive/receipt recovery. An earlier concurrent-source trial was discarded. No active Goal was promoted and no private source data is committed.
  • The frozen mutation runner now carries and hashes its required Python-discovery helper. Both affected baseline controls pass and both mutants are killed by assertions; missing imports are not counted as semantic failures.
  • TypeScript typecheck, strict Python mypy, changed-path Ruff, docs governance and public/private scan pass.
  • Scoped premerge passes 19 selected checks on the final committed head with one advisory: the unrelated Lark goal_topic_runtime.py maintainability debt also reproduces on clean main at d64c4d377. Its ceiling is unchanged; this is not an all-green broad-suite claim.
  • Exact-scope quality receipt is recorded and verified; details follow in the final-head validation comment.

Tradeoffs: complete parsed objects remain in memory; this is bounded bulk transport, not streaming or storage qualification. A 32-pair warm small-source comparison measured inline/artifact median 9.15/11.62 ms and p95 11.60/15.93 ms. A timed-out handler can leave a temporary file on Windows when the OS still holds it open. Frontend/API shapes and configuration are unchanged; existing public CLI entrypoints exercise the new transport. The managed-worker test now declares its intended selected-work acceptance scope explicitly.

Future-facing pass: use one domain-owned transfer codec across all affected hops, retain the existing typed semantic owners, and replace contradictory current roadmap counts with one evidence inventory. Source capture beyond this cap, event-writer binding, external-effect fencing, D2 qualification and default onboarding remain explicit boundaries. Maintainer merge required.

…nsfers

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
…er work

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
…ource adapter

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
@huangruiteng

huangruiteng commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Author validation for head d7ff0e2fa4921b9f02a9e47e39d0ea896fdef5bc (base d64c4d377).

The final diff is 21 files, +672/-106. Source projection and management use the existing typed domain handlers; Python retains byte/file adaptation in the existing source adapter. Wire identities and the 16 MiB artifact cap are generated from the coordination manifest. The generic 2 MiB RPC limit, source witnesses, durable operation identity, event-writer hold and provider/default authority are unchanged. The socket fix keeps a disconnect local to that connection; it does not cancel or replay the underlying operation.

Validation on the final implementation:

  • 118 Python source/capture/runtime/File/SQLite and public CLI lifecycle tests, plus 21 contract tests: passed.
  • 38 TS projection/transfer/dispatch tests plus 12 generated-contract tests: passed. Generation check and semantic-vocabulary drift check pass; independent Python/TS twins remain 43/43 without raising the budget.
  • TypeScript typecheck, strict Python mypy, focused Ruff, docs governance and public/private scan: passed.
  • Disposable PostgreSQL 16.15 conformance: 293 passed, no skips. Final real-source replay on a witness-verified detached copy passed bootstrap/inspect, complete File/SQLite readback and legacy/File/PostgreSQL archive/receipt recovery. No active Goal was promoted; private contents and raw traces are not published. The earlier concurrent live-source trial was discarded rather than counted as evidence.
  • Scoped premerge: 19 selected checks, zero blocking failures and zero manual holds. One advisory remains: the unchanged Lark goal_topic_runtime.py metric finding reproduces on clean main; its ceiling was not changed. This is not a claim that the repository-wide maintainability smoke is green.
  • Quality receipt cqr_7e460873f2f5468ba15b: valid for fingerprint 7e460873f2f5468ba15baf1b9d878f5b3b16ba005ea1cfc7cf9a58e9b743e480, with no blockers and one documented efficiency warning. Full source objects still occupy memory, and Windows may defer temporary-file cleanup after an ambiguous timeout.

The native vocabulary gate caught the initial additional same-name Python/TS module pair. The final implementation removes that pair, reuses the existing Python source adapter and generates the shared protocol instead of changing a ceiling. The future-facing pass also replaces contradictory current RFC PR counts with one inventory distinguishing shipped code, open PRs, proposed code boundaries and qualification work.

Frontend/API/configuration shapes are unchanged; the affected production CLI journey is exercised through independent processes. The remote mutant job also exposed a missing source-copy dependency on the already-shipped Python-discovery helper. The frozen runner now includes and hashes that helper; both unchanged controls pass and both injected defects fail by assertion. Broader D1–D3 qualification, event-writer binding, external-effect fences and default onboarding remain unclaimed. This control-plane PR is left for maintainer review and merge; GitHub CI is separate from the local evidence above.

…cles

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
@huangruiteng
huangruiteng merged commit ba8aafb into main Sep 24, 2026
21 of 29 checks passed
@huangruiteng
huangruiteng deleted the codex/source-projection-transport-0924 branch September 24, 2026 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant