From d922c4289b99d2218b90f4104f39475a72448282 Mon Sep 17 00:00:00 2001 From: Jonathan Fontanez Date: Sat, 22 Aug 2026 15:23:45 -0400 Subject: [PATCH 1/2] =?UTF-8?q?docs(plan):=20ship=20python-kernel-surface?= =?UTF-8?q?=20=E2=80=94=20fold=20into=20=C2=A7Graphics=20and=20archive?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every ticket of the change is merged: #1773 (named N-binding compute dispatch), #1774 (CPU readback as an always-present capability), #1775 (GLSL source contract), #1776 (batched dispatch), #1777 (graphics and ray-tracing parity), #1778 (cross-process texture import) and #1779 (the device-tensor scope), the last landing as PR #1902 on 2026-08-22. The REMOVED gate is clean — 13 bullets, none referenced and none on disk — so the four bridge traits, their installers, the nine bridge-absent paths, the v1 single-output convention and the three Python refusals are gone from the tree, not merely unused. Six of §Graphics' seven [python-kernel-api] entries are marked SHIPPED with verify markers. The seventh, the Rust bindings-at-dispatch convergence, is out of scope by the change's own sequencing and stays unmarked, so the section stays IN-FLIGHT — now driven by kernel-kind-parity-bar alone. Four facts the implementation established fold into the entries that own them, each an owner-approved consequence rather than a new decision: - The device-write scopes discard on the exception path (owner, 2026-08-07) — one rule for both, because blitting a half-written view back publishes a torn frame that surfaces downstream instead of at the raise. - A write-back is an edit of a frame the processor read; the engine refuses one into a staging no read of that frame landed in, since it cannot tell a consumer's write from uninitialised memory. - Cross-process importability is an allocation flavour the engine derives per acquisition, never a Python dial (approved at #1778's announce gate). - The batch sends one op on leaving the scope rather than holding the privileged gate open across user Python, and carries the two constraints that retire with the Rust convergence — one kernel once per batch, and no surface bound at two kinds in one dispatch. The named-binding error contract joins the kernel-as-object entry: every mismatch raises before any GPU work is submitted, naming the shader's own bindings, checked engine-side so the wheel is never the only guard. No plan text is retired — the REMOVED bullets retire code the plan already describes as deleted. The system diagram's helper→engine edge carries the kernel contract instead of the bare "kernels + texture handles" label. kernel-kind-parity-bar.md is corrected in the same PR, being the file this fold falsifies: its verbatim quote of the parity entry and its paste-ready replacement both carry the SHIPPED clause and markers forward (it narrows the claim, it does not unship it); its fifth-refusal note becomes the record that #1778 discharged it; and its fourteen ARCHITECTURE.md anchors are re-pointed. Those anchors were already stale by +28 lines from #1903's raw-handle entries before this fold moved them again. Refs #1773, #1774, #1775, #1776, #1777, #1778, #1779 Co-Authored-By: Claude Opus 5 (1M context) --- docs/plan/ARCHITECTURE.md | 64 +++++++++++++--- .../2026-08-22-python-kernel-surface.md} | 0 docs/plan/changes/kernel-kind-parity-bar.md | 75 ++++++++++++------- docs/plan/diagrams/system.mmd | 4 +- 4 files changed, 105 insertions(+), 38 deletions(-) rename docs/plan/changes/{python-kernel-surface.md => archive/2026-08-22-python-kernel-surface.md} (100%) diff --git a/docs/plan/ARCHITECTURE.md b/docs/plan/ARCHITECTURE.md index e8394c6a9..1cc02ffc7 100644 --- a/docs/plan/ARCHITECTURE.md +++ b/docs/plan/ARCHITECTURE.md @@ -212,7 +212,7 @@ Legend: **DECIDED** — build exactly this. **OPEN** — do not build; needs an green-thread style): intended, do not build until designed; hard constraint — no new configuration dials. [execution-model] -## Graphics (RHI / GPU) — IN-FLIGHT (→ python-kernel-surface, kernel-kind-parity-bar) +## Graphics (RHI / GPU) — IN-FLIGHT (→ kernel-kind-parity-bar) - **DECIDED** — All Vulkan lives in the RHI (`vulkan/rhi/` + `streamlib-consumer-rhi`); one kernel abstraction per pipeline kind; consumers go through `GpuContext` only. @@ -223,35 +223,81 @@ Legend: **DECIDED** — build exactly this. **OPEN** — do not build; needs an - **DECIDED** — Python reaches every GPU capability Rust authoring reaches: compute, graphics and ray-tracing kernels, acceleration structures, and CPU readback. Python names and drives; the engine allocates, compiles, binds, and dispatches. No kernel - capability is Rust-only. [python-kernel-api] + capability is Rust-only. [python-kernel-api; python-kernel-surface — SHIPPED #1773, + #1774, #1777] + + + + - **DECIDED** — A kernel's output is an engine-owned texture that Python names by surface id and passes downstream in a bag, and that a third-party GPU library in its own Python package reaches through a scope: entering blits the texture to a linear view (DLPack over DMA-BUF / OPAQUE_FD), leaving blits any write back and orders it on the surface's timeline ahead of the engine's next read. The engine owns that - ordering — no fence or timeline vocabulary reaches Python. Cross-process texture - import is part of the capability. [python-kernel-api] + ordering — no fence or timeline vocabulary reaches Python. Leaving the scope by a + propagating exception discards the write instead: a half-written view blitted back + publishes a torn frame that surfaces as corrupt pixels somewhere downstream rather + than at the `raise`, so the engine keeps the complete frame it already holds and lets + the exception propagate — one rule for both device-write scopes, the CPU pixel-buffer + scope included, and discarding never suppresses the exception. A write-back is always + an edit of a frame the processor read, never a fresh-frame write: the engine refuses + one into a staging no read of that frame landed in, because it cannot tell a + consumer's write from uninitialised memory and one staging spans every frame its pool + slot publishes. Cross-process texture import is part of the capability, and + importability is an allocation flavour the engine derives per acquisition, never a + Python dial: render-attachment usage with a probed DRM modifier takes + explicit-modifier DMA-BUF, a CUDA-mappable format within the OPAQUE_FD usage set + takes OPAQUE_FD, and everything else keeps a non-importable allocation whose later + cross-process import refuses by name. + [python-kernel-api; python-kernel-surface — SHIPPED #1778, #1779] + + + - **DECIDED** — Python spells a kernel as an object: constructed in `setup()` where the capability typestate is Full, dispatched per frame in `process()`. Construction is registration and dispatch is a method call; no kernel handle string reaches Python. Compute takes a general N-binding array like graphics and ray tracing — a Python - compute kernel reads one surface and writes another, at parity with Rust. - [python-kernel-api] + compute kernel reads one surface and writes another, at parity with Rust. A binding + mismatch raises before any GPU work is submitted, and the message names the shader's + declared bindings: an undeclared name, an unsupplied one, a name supplied twice and a + kind mismatch are refused at dispatch — the kernel holds no binding state, so there is + no implicit default and no carried-over value — while a stage mismatch and + name-stripped SPIR-V on the escape hatch are refused at construction. Every refusal is + checked engine-side, so the wheel is never the only guard. + [python-kernel-api; python-kernel-surface — SHIPPED #1773, #1777] + + - **DECIDED** — Compute, graphics, ray tracing, and CPU readback are always-present capabilities of `GpuContext`, reached the same way by every caller. The four bridge traits and their installation step are deleted: no kernel capability can be absent at - runtime, and no application glue supplies one. [python-kernel-api] + runtime, and no application glue supplies one. + [python-kernel-api; python-kernel-surface — SHIPPED #1773, #1774, #1777] + - **DECIDED** — GLSL is the shader source contract: Python passes GLSL text and the engine compiles it at kernel construction, and re-creating an identical kernel is free — compilation is cached under a key covering everything that changes the output (source, stage, entry point, target environment, compiler version), never source alone. Pre-compiled SPIR-V stays accepted as an escape hatch. Authoring a kernel requires no toolchain beyond the installed wheel, for every kernel kind. The wheel carries a C++ - GLSL compiler (shaderc / glslang). [python-kernel-api] + GLSL compiler (shaderc / glslang). [python-kernel-api; python-kernel-surface — + SHIPPED #1775] + + - **DECIDED** — Dispatch is synchronous: it returns when the GPU work has retired and the writes are visible, and no fence or timeline vocabulary reaches Python. Several dispatches batch into one submission with barriers between them and a single fence at - the end — the Python equivalent of the command-recorder flow. [python-kernel-api] + the end — the Python equivalent of the command-recorder flow. The batch accumulates its + dispatches and sends them as one op on leaving the scope, never holding the privileged + gate open across user Python; a raise inside the scope sends nothing. Two constraints + ride it while bindings still stash on the kernel, both refused by name and both + retiring with the Rust convergence below: one kernel may appear only once per batch, + because a kernel owns a single descriptor set and a second bind would silently hand the + earlier dispatch the later one's bindings; and one surface may not be bound at two + kinds in a single dispatch, because no image layout satisfies both a sampled and a + storage descriptor. [python-kernel-api; python-kernel-surface — SHIPPED #1773, #1776] + + + - **DECIDED** — One kernel spelling in both languages: bindings are passed at dispatch, by name, and never persist on the kernel object. Rust's stateful numeric-slot setters go; the command-recorder flow keeps its seam by carrying bindings to the recorder diff --git a/docs/plan/changes/python-kernel-surface.md b/docs/plan/changes/archive/2026-08-22-python-kernel-surface.md similarity index 100% rename from docs/plan/changes/python-kernel-surface.md rename to docs/plan/changes/archive/2026-08-22-python-kernel-surface.md diff --git a/docs/plan/changes/kernel-kind-parity-bar.md b/docs/plan/changes/kernel-kind-parity-bar.md index 4e4e97699..4f71357a2 100644 --- a/docs/plan/changes/kernel-kind-parity-bar.md +++ b/docs/plan/changes/kernel-kind-parity-bar.md @@ -20,20 +20,20 @@ supersede-in-place form, and a companion ADR would restate decision 1 in a secon clause of it. Recon verified at HEAD `648478f7` on 2026-08-17. Precondition satisfied, per entry: the parity -entry (ARCHITECTURE.md:195-198) carries `**DECIDED**`, as does the always-present-capabilities -entry (ARCHITECTURE.md:212-215), named only so the fold does not touch it. §Graphics' trailing -`- **OPEN** — Everything else.` (ARCHITECTURE.md:232) is a catch-all predating the align that did +entry (ARCHITECTURE.md:223-231) carries `**DECIDED**`, as does the always-present-capabilities +entry (ARCHITECTURE.md:270-275), named only so the fold does not touch it. §Graphics' trailing +`- **OPEN** — Everything else.` (ARCHITECTURE.md:306) is a catch-all predating the align that did not stop `python-kernel-surface` being proposed against this section; this change enumerates inside it and builds nothing against it. ## Behavior after this change -The parity bar is **every kernel kind** — already §Graphics' vocabulary (ARCHITECTURE.md:190, -:221) and GLOSSARY's kind-shaped **Kernel** (GLOSSARY.md:89-91). The plan stops promising Python +The parity bar is **every kernel kind** — already §Graphics' vocabulary (ARCHITECTURE.md:218, +:281) and GLOSSARY's kind-shaped **Kernel** (GLOSSARY.md:89-91). The plan stops promising Python can express every pipeline Rust can, and the four capabilities it cannot are spelled into the plan below. Two sentences must not move with it, because "no kernel capability" appears twice in -§Graphics with two meanings: ARCHITECTURE.md:214-215 is about the deleted bridge traits and -*runtime* absence, not language parity, and ARCHITECTURE.md:209-210 — "a Python compute kernel +§Graphics with two meanings: ARCHITECTURE.md:272-273 is about the deleted bridge traits and +*runtime* absence, not language parity, and ARCHITECTURE.md:260 — "a Python compute kernel reads one surface and writes another, at parity with Rust" — is a shipped claim that stays true. A find-and-replace narrowing would gut the first; neither is amended. @@ -110,18 +110,27 @@ path for any current consumer; the v1 surface deliberately rejects sample count ## MODIFIED - MODIFIED: §Graphics, the Python-parity DECIDED entry - ARCHITECTURE.md:195-198. Today, verbatim: + ARCHITECTURE.md:223-231. Today, verbatim: ```markdown - **DECIDED** — Python reaches every GPU capability Rust authoring reaches: compute, graphics and ray-tracing kernels, acceleration structures, and CPU readback. Python names and drives; the engine allocates, compiles, binds, and dispatches. No kernel - capability is Rust-only. [python-kernel-api] + capability is Rust-only. [python-kernel-api; python-kernel-surface — SHIPPED #1773, + #1774, #1777] + + + + ``` + The SHIPPED clause and its four markers arrived with `python-kernel-surface`'s fold on + 2026-08-22 and are carried into the replacement below — this change narrows the claim, it + does not unship it. + The exact replacement, paste-ready. The enumeration survives verbatim because it is what the ruling narrows *to*, and so does "Python names and drives". Gaps 1 and 3 are named as a clause - inside the entry — the shape of the Apple-capture gap at ARCHITECTURE.md:246-248, putting the + inside the entry — the shape of the Apple-capture gap at ARCHITECTURE.md:320-322, putting the absence where the claim is read: ```markdown @@ -135,14 +144,19 @@ path for any current consumer; the v1 surface deliberately rejects sample count Rust consumers in the engine tree hold them, and the only by-surface-id resolution the escalate path has is texture-shaped, so a Python processor is refused by name. Both are undesigned. - [python-kernel-api; kernel-kind-parity-bar — the parity claim narrowed to kernel kinds] + [python-kernel-api; python-kernel-surface — SHIPPED #1773, #1774, #1777; + kernel-kind-parity-bar — the parity claim narrowed to kernel kinds] + + + + ``` Plan prose is rewritten, never struck — ARCHITECTURE.md holds no strikethrough anywhere — and the bracket's semicolon clause is the document's idiom for a partial later amendment - (ARCHITECTURE.md:77-78, :296-297), applied with the markers at the ship fold. + (ARCHITECTURE.md:77-78, :370-371), applied with the markers at the ship fold. **RESOLVED (owner, 2026-08-17)** — the gap does not block MVP on its own terms, and the one case it would block is reopened through a texture rather than a buffer. Of the four, only gap 3 @@ -167,8 +181,8 @@ path for any current consumer; the v1 surface deliberately rejects sample count plus a buffer arm on binding resolution, the largest of the three engine changes. - MODIFIED: §Graphics, the trailing OPEN entry - ARCHITECTURE.md:232, today the whole line `- **OPEN** — Everything else.` — expanded to name - gaps 2 and 4, the shape §Networking's OPEN entry uses at ARCHITECTURE.md:286. They land here + ARCHITECTURE.md:306, today the whole line `- **OPEN** — Everything else.` — expanded to name + gaps 2 and 4, the shape §Networking's OPEN entry uses at ARCHITECTURE.md:360. They land here rather than inside the DECIDED entry because neither is a Python-reach gap: ```markdown @@ -245,21 +259,26 @@ acquired texture, and one post-MVP ticket for gap 1 on `Graphics Kernel Buildout ## Notes (not tickets) -- **The section flip is done; the fold is per-entry.** §Graphics' header now reads - `IN-FLIGHT (→ python-kernel-surface, kernel-kind-parity-bar)` (ARCHITECTURE.md:187), the - comma-list form §Media I/O uses at ARCHITECTURE.md:234. It cannot flip to SHIPPED while - `python-kernel-surface` is live, so this fold marks only the two entries it names — as - ARCHITECTURE.md:77-78 carries a per-clause SHIPPED note under an IN-FLIGHT §Packages header. - -- **A fifth refusal exists and is not a fifth disposition.** "importing a foreign DMA-BUF is not - reachable from a Python processor yet" is still live at `python_processor_context.rs:1200`, - against a plan sentence promising "Cross-process texture import is part of the capability" - (ARCHITECTURE.md:204-205). It is an open removal bullet of the in-flight - `python-kernel-surface` change (`python-kernel-surface.md:170`) — unfinished work with an - owner, not a permanent gap. +- **The section flip is done; the fold is per-entry.** §Graphics' header reads + `IN-FLIGHT (→ kernel-kind-parity-bar)` (ARCHITECTURE.md:215) — `python-kernel-surface` + archived 2026-08-22 and left the comma-list form §Media I/O still uses at + ARCHITECTURE.md:308. It cannot flip to SHIPPED even so: §Graphics' last + `[python-kernel-api]` entry, the Rust bindings-at-dispatch convergence + (ARCHITECTURE.md:301-305), is unbuilt and has no change file. This fold therefore marks only + the two entries it names — as ARCHITECTURE.md:77-78 carries a per-clause SHIPPED note under + an IN-FLIGHT §Packages header. + +- **The fifth refusal was never a fifth disposition, and is now gone.** "importing a foreign + DMA-BUF is not reachable from a Python processor yet" was live at + `python_processor_context.rs:1200` when this change was written, against a plan sentence + promising "Cross-process texture import is part of the capability" (ARCHITECTURE.md:246-247). + It was a removal bullet of `python-kernel-surface`, discharged by #1778 and proved absent by + that change's ship gate on 2026-08-22 + (`archive/2026-08-22-python-kernel-surface.md`) — unfinished work with an owner, exactly as + recorded, never a permanent gap. - **§Language SDKs & parity is untouched and owes this change nothing.** Its entries - (ARCHITECTURE.md:290-315) state no capability-parity claim. The sentence this change narrows + (ARCHITECTURE.md:364-389) state no capability-parity claim. The sentence this change narrows does not carry the word `parity` at all — it had to be found by reading §Graphics — and the one - place ARCHITECTURE.md uses the word for a claim, :210, is the shipped compute claim that stays + place ARCHITECTURE.md uses the word for a claim, :260, is the shipped compute claim that stays true. GLOSSARY.md defines neither `parity` nor `capability`. \ No newline at end of file diff --git a/docs/plan/diagrams/system.mmd b/docs/plan/diagrams/system.mmd index b45aa0e54..a651c6359 100644 --- a/docs/plan/diagrams/system.mmd +++ b/docs/plan/diagrams/system.mmd @@ -28,7 +28,9 @@ flowchart LR engine -->|"bags carry per-frame surface ids (slot#generation); claim at typed cast, lease until drop; a stale id errors, never another frame"| helper - helper -->|kernels + texture handles| engine + helper -->|"kernels constructed in setup(), dispatched by name and +synchronously — one submission per batch; a kernel output leaves +as a DLPack tensor scope whose write blits back, engine-ordered"| engine ctl --> registry rustapp --> crate crate --> engine From b2e5a8d3b67e6104c1bb1544d58da1123d4ba2ef Mon Sep 17 00:00:00 2001 From: Jonathan Fontanez Date: Sat, 22 Aug 2026 15:41:14 -0400 Subject: [PATCH 2/2] docs(plan): tighten the folded importability, write-back and diagram wording Three review findings on the fold, each verified against the tree before acting: - The importability summary dropped two conditions the derivation actually applies (`derive_texture_cross_process_importability`, subprocess_escalate.rs:3990-4017): DMA-BUF needs a single-plane format as well as a probed modifier, and OPAQUE_FD needs the image pool present. As written the plan overstated when an acquisition is importable. The fallback-not-failure rule the function documents is stated too. - The write-back precondition named its subject by pronoun. It now names the refused operation directly. - The diagram label collapsed two distinct things. A kernel output is named by surface id in a downstream bag; the DLPack scope over it is the optional third-party read / write-back, not how every output leaves. Co-Authored-By: Claude Opus 5 (1M context) --- docs/plan/ARCHITECTURE.md | 16 +++++++++------- docs/plan/diagrams/system.mmd | 4 ++-- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/plan/ARCHITECTURE.md b/docs/plan/ARCHITECTURE.md index 1cc02ffc7..62ba2a68c 100644 --- a/docs/plan/ARCHITECTURE.md +++ b/docs/plan/ARCHITECTURE.md @@ -241,14 +241,16 @@ Legend: **DECIDED** — build exactly this. **OPEN** — do not build; needs an the exception propagate — one rule for both device-write scopes, the CPU pixel-buffer scope included, and discarding never suppresses the exception. A write-back is always an edit of a frame the processor read, never a fresh-frame write: the engine refuses - one into a staging no read of that frame landed in, because it cannot tell a - consumer's write from uninitialised memory and one staging spans every frame its pool - slot publishes. Cross-process texture import is part of the capability, and + a write-back into a staging that has not first read that same frame, because it cannot + tell a consumer's write from uninitialised memory and one staging spans every frame its + pool slot publishes. Cross-process texture import is part of the capability, and importability is an allocation flavour the engine derives per acquisition, never a - Python dial: render-attachment usage with a probed DRM modifier takes - explicit-modifier DMA-BUF, a CUDA-mappable format within the OPAQUE_FD usage set - takes OPAQUE_FD, and everything else keeps a non-importable allocation whose later - cross-process import refuses by name. + Python dial: single-plane render-attachment usage takes explicit-modifier DMA-BUF + where the render-target modifier probes available; a CUDA-mappable format whose usage + sits inside the OPAQUE_FD set takes OPAQUE_FD where that image pool exists; everything + else keeps a non-importable allocation. A flavour the device or format cannot take + falls back rather than failing the acquire, and the later cross-process import refuses + by naming the flavour. [python-kernel-api; python-kernel-surface — SHIPPED #1778, #1779] diff --git a/docs/plan/diagrams/system.mmd b/docs/plan/diagrams/system.mmd index a651c6359..9368091b3 100644 --- a/docs/plan/diagrams/system.mmd +++ b/docs/plan/diagrams/system.mmd @@ -29,8 +29,8 @@ flowchart LR claim at typed cast, lease until drop; a stale id errors, never another frame"| helper helper -->|"kernels constructed in setup(), dispatched by name and -synchronously — one submission per batch; a kernel output leaves -as a DLPack tensor scope whose write blits back, engine-ordered"| engine +synchronously — one submission per batch; an output is named by surface id, +and a DLPack scope over it is the optional third-party read / write-back"| engine ctl --> registry rustapp --> crate crate --> engine