Skip to content

feat(ffi): expose SHM provider through the FFI (+ hiroz-go WithShmPool) - #345

Open
richardw347 wants to merge 2 commits into
ZettaScaleLabs:mainfrom
richardw347:feat/ffi-shm-provider
Open

richardw347 wants to merge 2 commits into
ZettaScaleLabs:mainfrom
richardw347:feat/ffi-shm-provider

Conversation

@richardw347

Copy link
Copy Markdown
Contributor

Summary

The Rust API supports shared memory — ZContextBuilder::with_shm_pool_size() /
with_shm_config() and the whole shm module — but the C FFI never surfaced
it
. hiroz_context_config_t has no SHM field and
hiroz_context_create_with_config never attaches a provider, so no FFI
consumer (Go/C/Python) can make a session SHM-capable.

This matters because in zenoh 1.9 a session only negotiates SHM transport with a
peer if it has an SHM provider attached. Setting
transport/shared_memory/enabled: true via config alone is not sufficient — a
provider-less peer is treated as non-SHM and the router copies payloads out over
the transport. So an FFI consumer receiving large payloads (e.g. camera frames)
from an SHM-capable Rust publisher silently falls back to a full copy over TCP,
even with SHM "enabled".

Changes

  • CContextConfig: add shm_pool_bytes: usize (appended after existing
    fields → ABI-additive). When non-zero, hiroz_context_create_with_config
    attaches a provider via builder.with_shm_pool_size(cfg.shm_pool_bytes),
    mirroring the Rust builder; 0 disables (default). Regenerated hiroz_ffi.h.
  • hiroz-go: add ContextBuilder.WithShmPool(bytes uint64) and marshal it
    into the config struct.

Both mirror existing patterns (with_shm_pool_size already exists in the core;
the FFI error handling mirrors the with_remap_rules block).

Sizing note for consumers

A receiver needs only a small pool — it maps the publisher's segments and
allocates nothing per message; the pool exists solely to make the session
SHM-capable. A publisher needs a pool large enough for its in-flight payloads.

Verification

  • cargo build --features ffi compiles the FFI change and regenerates
    hiroz_ffi.h with uintptr_t shm_pool_bytes; in hiroz_context_config_t.
  • hiroz-go builds and links WithShmPool against the resulting staticlib.

Provenance

Found consuming hiroz from Go (cgo) on a robot: a camera driver (Rust,
SHM-capable) published 20MP frames into SHM, but the Go consumer — unable to
attach a provider through the FFI — received them over TCP loopback (measured
~780 MB/s on lo during capture where SHM would be ~KB/s). This PR is the
missing FFI capability, not a workaround.

The Rust API supports shared memory (ZContextBuilder::with_shm_pool_size /
with_shm_config), but the C FFI never surfaced it, so no FFI consumer (Go/C/
Python) could make a session SHM-capable. In zenoh 1.9 a session only
negotiates SHM transport if it has an SHM provider attached; setting
transport/shared_memory/enabled alone is not sufficient, so an FFI consumer's
large payloads always fall back to a copy over the transport.

- CContextConfig: add shm_pool_bytes (appended; ABI-additive). When non-zero,
  hiroz_context_create_with_config attaches a provider via with_shm_pool_size,
  mirroring the Rust builder; 0 disables (default). Regenerated hiroz_ffi.h.
- hiroz-go: add ContextBuilder.WithShmPool(bytes) and marshal it into the config
  struct.

Verified: hiroz builds with --features ffi and regenerates the header with the
new field; hiroz-go builds and links WithShmPool against the resulting staticlib.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@richardw347 richardw347 changed the title feat(ffi): expose SHM provider through the FFI (+ hiroz-go WithShmPool) feat(ffi): expose SHM provider through the FFI (+ hiroz-go WithShmPool) Sep 13, 2026
Resolve hiroz_ffi.h conflict by keeping shm_pool_bytes from this branch.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant