plan(python): spell CPU readback and write-back of texture-backed surfaces in the wheel

[tato123]

Re-scoped by /reconcile-tracker 2026-08-08. The kernel half of this ticket is superseded — #1773 (compute), #1775 (GLSL), #1776 (batching) and #1777 (graphics + ray tracing) deliver it, and the bridge-installation approach this body proposed is superseded by ADR decision 5 (docs/decisions/python-kernel-api.md), which deletes the bridge traits entirely rather than installing them from the wheel's spawn path. Every [NEEDS DECISION] in this ticket's own comment now has an ADR answer.

The cpu-readback half survives, and it is narrower than this body implies.

What already works — no ticket owed

A Python author CPU-reads a pixel-buffer-backed surface today with zero glue. lock() / as_numpy() route through host_visible_pixel_plane() (sdk/streamlib-python-wheel/src/python_gpu_surface_pixel_exchange.rs:113-122) — these are host-visible linear allocations, so no GPU→CPU copy runs at all. The parity bar is already discharged here.

The actual gap

A numpy-only author cannot get a texture-backed surface — a kernel output — onto the CPU.

lock() raises "surface has no host mapping; it is a DEVICE_LOCAL allocation" (src/python_processor_context.rs:317-330); the host DLPack arm fails the same way (:416-427) and as_numpy requests device="cpu" (:465-468), landing on it. The only working path is a CUDA capsule over the OPAQUE_FD device-export staging, which needs a live CUDA runtime plus torch/cupy to copy device→host — numpy.from_dlpack cannot consume a CUDA capsule. #1779's as_device_tensor() does not close this: it delivers a device tensor for third-party GPU packages, not CPU bytes.

This gap only becomes reachable once #1773/#1777 let a Python author produce kernel outputs, and #1774 gives the engine the readback capability. So it is real, and it is new.

Why this is design-first, not an implementation ticket

The plan states the capability (§Graphics: "Python reaches every GPU capability Rust authoring reaches … and CPU readback") but no plan entry states the spelling. There is a genuine fork:

• Extend GpuSurfaceHandle's host arm so a texture-backed surface refills a host-mappable staging and as_numpy() simply works on kernel outputs — the one-system shape, likely over the existing device-export staging.
• Or a dedicated readback vocabulary, resurrecting the legacy acquire_read / acquire_write / try_acquire_* context managers (sdk/streamlib-python/python/streamlib/adapters/cpu_readback.py:487-520).

Either way the contended wire response (escalate_request.yaml:143-148, handler subprocess_escalate.rs:1112-1146) needs a non-exception Python shape — in the legacy vocabulary it yielded None. That is an architecture decision; tickets implement the plan, they never make it.

Depends on #1774 (the engine capability) and #1773 (something to read). Its resolution may ADD a bullet to docs/plan/changes/python-kernel-surface.md or ride a small follow-up change — an /align call, not an implementer's.

Original body (pre-reconciliation, preserved 2026-08-08)

Named disposition from #1714's escalate-parity research (owner, 2026-08-06). Three of three,
and the largest.

A Rust processor can build and dispatch compute, graphics and ray-tracing kernels and build
acceleration structures. A Python processor cannot spell any of it — not because the path is
missing, but because the wheel never grew the vocabulary.

What already exists

Every op is defined and served. packages/escalate/schemas/escalate_request.yaml carries
register_compute_kernel, run_compute_kernel, register_graphics_kernel, run_graphics_draw,
register_acceleration_structure_blas, register_acceleration_structure_tlas,
register_ray_tracing_kernel, run_ray_tracing_kernel, run_cpu_readback_copy and
try_run_cpu_readback_copy; handle_escalate_op implements all of them; and the wheel's spawn
host already attaches the bridge that routes a helper's escalate frames into that handler
(python_helper_process_spawn_host.rs:484-488). The old subprocess SDK drove them through
adapter classes — sdk/streamlib-python/python/streamlib/adapters/vulkan.py is the template,
and examples/polyglot-vulkan-{compute,graphics,ray-tracing} are the live consumers.

What is missing

1. Wheel-side spelling — the Python surface a processor calls.
2. Bridge installation. The four kernel/readback domains are trait-dispatched and their
   implementations are installed by application setup glue today
   (core/context/compute_kernel_bridge.rs:21-27; e.g.
   examples/polyglot-vulkan-compute/src/main.rs:298). Without an installed bridge the host
   answers a typed err. Under the importable-library model there is no app main.rs to do it,
   so the wheel's spawn path has to.

Shape constraint

The subprocess-Vulkan carve-out stands (.claude/rules/polyglot.md): import-side only —
no allocation, no modifier choice, no kernel construction child-side. Everything privileged
escalates. ARCHITECTURE.md §Graphics already decides the surface: "the engine's kernel
primitives are exposable to Python as configured blocks: shader/compute source and binding
config passed from Python, compiled and executed by the engine on its device — no user-side
Vulkan, ever. Shape only; the Python-facing API design is its own session."

That last clause means this needs an API design pass before implementation.

[tato123]

[NEEDS DECISION] — the plan defers this ticket's core design by name

/implement plan gate. ARCHITECTURE.md:143-146 decides the shape and then closes with
"Shape only; the Python-facing API design is its own session." No change file under
docs/plan/changes/ covers it (grep -rn "kernel" docs/plan/changes/ → nothing). This
ticket's own closing line and its shape:design-first label say the same. Stopping here.

Staleness check — the body holds up

Verified against the tree today; no corrections needed:

• All ten ops present in packages/escalate/schemas/escalate_request.yaml (compute :229/:262,
  graphics :311/:592, BLAS/TLAS :756/:786, ray tracing :856/:1000, readback :101/:134).
• handle_escalate_op implements them (compiler_ops/subprocess_escalate.rs).
• The wheel's spawn host attaches the bridge — now python_helper_process_spawn_host.rs:483-487,
  one line off from the body's 484-488.
• core/context/compute_kernel_bridge.rs:21-27 and examples/polyglot-vulkan-compute/src/main.rs:298
  are as described.

One thing the body could not have known: there is no reusable bridge implementation anywhere.
impl ComputeKernelBridge | impl CpuReadbackBridge | impl GraphicsKernelBridge | impl RayTracingKernelBridge
matches only the four examples/polyglot-* apps (all slated for deletion by the ripout's
dispositions) and the escalate handler's own test module. So "the wheel's spawn path has to
install it" has nothing to install — the wheel would have to author four impls from scratch.

The five decisions this needs

1. Surface shape. The plan says "configured blocks", which does not pick one.

• (a) A ctx.gpu namespace mirroring the old VulkanContext — dispatch_compute,
  dispatch_graphics, build_blas, build_tlas, dispatch_ray_tracing, plus readback.
  Closest to the wire; sdk/streamlib-python/python/streamlib/adapters/vulkan.py:531-877 is
  the exact template.
• (b) Kernel objects constructed at setup, dispatched per frame —
  ComputeKernel(source=..., push_constant_size=...) in setup(), kernel.dispatch(...) in
  process(). The wire is already register-once-dispatch-many, so this maps 1:1 and never
  exposes a handle string to Python.
• (c) Declarative — kernels declared in @processor metadata, registered by the engine at setup.

Recommendation: (b). It is the honest shape of the protocol, it keeps the two-phase cost
model visible (registration is expensive, dispatch is not), and it puts registration in
setup() where the capability typestate is Full. (a) invites per-frame re-registration;
(c) adds a declaration surface the pivot has been deleting everywhere else.

2. Shader source: SPIR-V or GLSL? "compiled and executed by the engine" is ambiguous.
The escalate ops take SPIR-V blobs, and glslc today is a build-time dependency only
(runtime/streamlib-engine/build.rs) — nothing in the wheel compiles a shader at runtime.
So either Python hands over pre-compiled SPIR-V (zero new deps, but the user needs a shader
toolchain — against the zero-ceremony bar), or the wheel ships a runtime compiler (shaderc
links ~a few MB into the wheel; weighs on the manylinux portability model in §Distribution).

Recommendation: accept both, GLSL-first in the docs — source= (GLSL, engine-compiled)
and spirv= (pre-compiled, escape hatch). But shipping shaderc in the wheel is a
§Distribution decision, not mine.

3. Where do the bridges live now?

• (a) The wheel authors and installs four impls at spawn — but that makes the wheel depend on
  whatever holds the Vulkan work, and re-creates the app-glue pattern the pivot removed.
• (b) Collapse the seam. The four bridge traits exist because there were two arms — a host
  arm and a cdylib arm. The cdylib arm dies with the ABI, and helper children always take the
  host arm (importable-python-library-ripout.md:109-116 already states the branch collapses).
  Make the four domains always-present capabilities of GpuContext — no trait, no
  installation, no typed err when absent.

Recommendation: (b), and it is the reason this is a plan question and not an in-ticket one:
deleting a core extension seam is architecture. It also removes a whole failure mode (a Python
processor asking for compute and getting "no bridge installed").

4. Scope — all four domains, or a subset? The MVP sentence is camera-in-a-window in a
minute. Acceleration structures and ray tracing are far from it. Compute + cpu-readback carry
most of the practical value (that is what polyglot-cpu-readback-blur and
polyglot-vulkan-compute actually did).

Recommendation: compute + cpu-readback in this ticket; graphics and ray-tracing as a named,
tracked follow-up — not a silent deferral, per the full-parity bar.

5. What does Python bind a kernel to? The ops bind by surface_uuid from
SurfaceStore::register_texture. The wheel's Python-facing frame is VideoFrame.surface_id
(video_frame.py:121). Does dispatch(...) take a VideoFrame, a bare surface_id string,
or a DLPack capsule (which §Packages names as the ecosystem exchange surface)?

Recommendation: take the VideoFrame (and a bare surface_id where there is no frame).
DLPack is the export contract for handing pixels to torch/cupy; it is the wrong currency for
naming an engine-side render target.

What I would ask for

An /align session on the Python kernel API, producing a change file — that is what
"its own session" in ARCHITECTURE.md:146 is asking for. If you would rather rule on the five
above directly in a comment here, that unblocks implementation just as well.

[tato123]

DEFERRED from #1774 — readback staging memory type (performance, not correctness)

Landing this here rather than only in #1774's PR body, because this ticket creates the first thing that will actually read that memory — so the constraint belongs where it cannot be missed.

What is wrong. #1774 makes CPU readback an always-present GpuContext capability. The staging it reads into is allocated by HostVulkanBuffer::new_opaque_fd_export, which draws from the OPAQUE_FD pool probed with HOST_ACCESS_SEQUENTIAL_WRITE (vulkan_device.rs::create_opaque_fd_buffer_pool). That hint tells VMA "the CPU only writes, sequentially", so it deliberately selects uncached / write-combined memory. CPU reads from write-combined memory are roughly an order of magnitude slower than from HOST_CACHED. The capability being shipped is a CPU read path.

Measured on the rig with vulkaninfo — both types exist, and the pool is landing on the uncached one:

type	flags
memoryTypes[3]	HOST_VISIBLE | HOST_COHERENT	uncached / write-combined ← selected
memoryTypes[4]	HOST_VISIBLE | HOST_COHERENT | HOST_CACHED	cached

Why it was not fixed in #1774.

• Not a defect feat(engine): CPU readback becomes an always-present GpuContext capability #1774 introduced. That allocator's memory-type choice predates the branch and serves paths where "CPU only writes" is the correct hint — including the CUDA-interop producer allocation (feat(plugin-sdk): OPAQUE_FD export buffer + CUDA adapter over the plugin ABI #1262, gpu_context.rs). feat(engine): CPU readback becomes an always-present GpuContext capability #1774 is simply the first thing to make CPU reads of it a headline capability.
• Not a flag change. VMA pins a custom pool's memory type at creation, so allocation flags are ignored afterward. It needs a third OPAQUE_FD pool, which means also deciding the fallback for hardware with no exportable cached type — and "refuse readback" as that fallback would be a functional regression versus what ships today, which works everywhere and is merely slower.
• This ticket's design may moot it. If the spelling settled here has the parent do the read and ship bytes over the wire, the staging may not need to be exportable at all, and a plain cached allocation with no pool suffices. Fixing it before this design lands risks fixing the wrong thing.
• It is rig-only-verifiable code that runs at device init for every GPU user, and CI never runs GPU tests.

What this ticket owes. Pick the fix shape once the Python spelling is decided — a third HOST_CACHED pool with a stated fallback, a re-probe of the shared pool weighed against the CUDA write path it also serves, or a non-exportable cached allocation if the read happens parent-side. Resolve before or with the first Python reader.

Does not block M39 — the MVP sentence is camera-in-a-window, readback is not on that path, and the capability is correct on every device today.

[tato123]

Widened by owner ruling, 2026-08-17 — this ticket now owns CPU write as well as read

Change: docs/plan/changes/kernel-kind-parity-bar.md (approved 2026-08-17), PR #1897.

Why the scope grew

That change narrows §Graphics' parity claim to kernel kinds and records the four capabilities
Python cannot reach as named dispositions. One of the four — storage_buffer / uniform_buffer
bindings — was the only one with a plausible MVP consumer, because push constants carry a filter's
knobs but not a data table: a colour LUT, a film curve, convolution weights.

The owner ruled that gap does not block MVP, on the condition that the data-table case is
reopened through a texture rather than a buffer — which is this ticket. So the LUT route now
depends on a Python author being able to write CPU bytes into an acquired texture, not just read
them out.

The read and write directions are one design question, not two, which is why this widens rather
than splitting into a second design-first ticket. The fork this ticket already states —
extend GpuSurfaceHandle's host arm versus a dedicated readback vocabulary — has to answer both
directions at once, and answering them separately is how the two scopes end up with two spellings.

What the ruling adds to the design question

• CPU write into an acquired (DEVICE_LOCAL, texture-backed) surface, so a numpy author can
  fill an rgba32_float LUT and bind it as a sampled_texture. Today lock() refuses it —
  "surface has no host mapping; it is a DEVICE_LOCAL allocation" (python_processor_context.rs:365).
• Whether write reuses the read spelling or needs its own. The engine-side write-back machinery is
  feat(python): the scoped device-tensor view over a kernel output #1779's (record_copy_buffer_to_image, texture-backed write-back); this ticket owns the Python
  spelling, so the two need to agree on who owns the round trip.
• The contended response still needs a non-exception Python shape, and it now applies to both
  directions.

Anchors refreshed 2026-08-17 — the body's have drifted

Verified against the tree at 3b4b9b0d, not inferred:

Body says	Now
escalate_request.yaml:143-148 (the contended response)	That file no longer exists. The wire is hand-written Rust: EscalateResponseContended at subprocess_escalate_wire_types/escalate_response.rs:25, variant at :14
subprocess_escalate.rs:1112-1146 (contended handler)	subprocess_escalate.rs:1192 emits it; the doc explaining when is at :1154
python_processor_context.rs:317-330 (the lock() refusal)	:365
:416-427 (host DLPack arm)	the same refusal now also lives at python_gpu_surface_pixel_exchange.rs:392, worded to name the device-export route
:465-468 (as_numpy requests device="cpu")	:510

sdk/streamlib-python/python/streamlib/adapters/cpu_readback.py — the legacy vocabulary this body
offers as the second horn of the fork — was deleted with the pre-pivot SDK (#1715). It is readable
in history only, and per CLAUDE.md it is reference material for what the wire could express, never
a target to build against.

Still true

shape:design-first still fits: no plan entry states the spelling, in either direction, so this
resolves through /align and not by an implementer picking one. Its blockers are discharged —
#1773 and #1774 both closed 2026-08-16 — so nothing gates the design work now.

[tato123]

Resolved — the design this ticket asked for is decided and landed

The /align session ARCHITECTURE.md §Graphics called for ran 2026-08-24. Every question this ticket and its comments carried has a recorded answer:

• The spelling, both directions (the body's fork + the 2026-08-17 widening): one door — the cast object's cpu() and the surface handle's lock()+as_numpy() work on texture-backed surfaces over the host-visible export staging; no readback vocabulary, no door names the backing. Read-in always runs on entry (pure writes — the LUT fill — included); a staged write publishes at the block edge and discards on a propagating raise. Decided entry: §Graphics [texture-backed-cpu-reach] (PR docs(plan): texture-backed CPU reach — one door over the host-visible staging #1938); rationale: docs/decisions/texture-backed-cpu-reach.md.
• The contended non-exception shape: dissolved rather than designed — every staging copy blocks, contended reaches no author, and the caller-less try_run_cpu_readback_copy / EscalateResponseContended / engine try_* surface is deleted (decision 4).
• The readback staging memory-type deferral (the feat(engine): CPU readback becomes an always-present GpuContext capability #1774 comment above): resolved by decision 6 — a third OPAQUE_FD pool, HOST_ACCESS_RANDOM probe, HOST_VISIBLE|HOST_COHERENT required with HOST_CACHED VMA-preferred, explicit post-probe check, degrade-never-refuse. Recon also surfaced that the staging registration must carry the exporter's vk_memory_type_index (textures already do) — mandatory, not hardening.
• Plus one the comments didn't anticipate: Python's acquire_texture implies copy_src+copy_dst (decision 7), so the LUT author writes usage=["texture_binding"] and the doors just work.

Delta: docs/plan/changes/texture-backed-cpu-reach.md (PR #1939, merged). Implementation is tracked by its own tickets, derived and edged 2026-08-24:

• feat(engine): the readback staging goes host-cached — a third OPAQUE_FD pool and the exporter's memory type on the staging wire #1940 — engine: the cached pool + the exporter's memory type on the staging wire
• chore(engine): delete the non-blocking readback surface — the try_ op, contended, and the contention enum #1941 — engine: delete the non-blocking readback surface (parallel)
• feat(python): the CPU doors open on texture-backed surfaces — staged read-in, block-edge publish, implied copy usages #1942 — wheel: the CPU doors open on texture-backed surfaces (blocked by feat(engine): the readback staging goes host-cached — a third OPAQUE_FD pool and the exporter's memory type on the staging wire #1940)

Nothing GPU-native is removed — writable(), as_device_tensor(), device-side DLPack, kernel dispatch and the raw exports are untouched; this adds the CPU path beside them.