Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
19a818d
feat(wheel)!: a Python processor's config is one class named by its _…
tato123 Sep 11, 2026
37fdbfb
refactor(packages)!: the extension wheels' four processors take a con…
tato123 Sep 11, 2026
a2f19a8
fix(wheel): the deriver sees both TypedDict spellings and names `Any`…
tato123 Sep 11, 2026
1158b4e
test(webrtc): the missing-endpoint refusal names its config class, no…
tato123 Sep 11, 2026
bc49159
test(wheel): the config-class tests type-check clean
tato123 Sep 11, 2026
8793694
test(wheel): a running node serves each config class's schema, GPU-free
tato123 Sep 11, 2026
3991513
docs(decisions): the extension wheels migrated in-stream, not as lagg…
tato123 Sep 11, 2026
13af247
fix(wheel): a self-referential config class stops instead of exhausti…
tato123 Sep 11, 2026
8197820
chore: drop an unrelated research memo swept in by a broad stage
tato123 Sep 11, 2026
9e74eec
docs(wheel): the deriver names the two spellings it does not share wi…
tato123 Sep 11, 2026
ab77941
test(wheel): a null default survives the msgpack hop into the descriptor
tato123 Sep 11, 2026
21a511f
style(wheel): rustfmt the config-schema reader's signature
tato123 Sep 11, 2026
f4dacf2
fix(wheel): the deriver describes four shapes it was silently dropping
tato123 Sep 11, 2026
1e2fae0
refactor(wheel): the config-schema reader refuses in its own vocabulary
tato123 Sep 11, 2026
ab40dcf
fix(wheel): the catalog end-to-end proof runs where a running graph c…
tato123 Sep 11, 2026
1f91bdd
test(wheel): the served rendering covers the null leg of the hop, and…
tato123 Sep 11, 2026
301477c
fix(wheel): the catalog states a tuple's length and only keys the con…
tato123 Sep 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions docs/decisions/agent-readable-processor-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,14 @@ the other way, code to document, and attaches to nothing on a link.
processor adds no dependency; three local config enums gain the derive.
- The descriptor carries a schema document where it carried a type-name string.
- Every Python processor that took keyword configuration changes shape: the engine-tree
fixtures migrate with the change; example and extension-wheel processors lag as
consumers do, and the extension wheels' required parameters become required keys in a
config class rather than defaults.
fixtures migrate with the change, and the extension wheels' required parameters become
required keys in a config class rather than defaults. ~~Example and extension-wheel
processors lag as consumers do~~ — amended 2026-09-11 by the owner's ruling that the
four extension-wheel processors migrate in the same PR as the engine half, as the
deliberate canary §Consumers reserves for in-flight work: `packages/` is the only
consumer tree with a CI lane, so migrating it is what proves the new construction path
on real processors and keeps that lane green. The fourteen example processors lag as
consumers do, unchanged.
- Reconfiguration takes the config object; `configure(self, **config)` goes with the
keyword form.
- A helper process importing a decorated class must register nothing, so the wheel has
Expand Down
15 changes: 10 additions & 5 deletions docs/decisions/importable-python-library.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,11 +198,16 @@ process boundary).
least one input port defaults; one declaring none must say what it is. A source has nothing to
react to, so the default would hand the author a processor that silently never runs — the one
case where the convenient default is a trap.
- **Configuration is constructor keyword arguments.** `rt.add(Blur, config={"radius": 3})`
constructs `Blur(radius=3)`, so a processor's settings are ordinary Python parameters with
ordinary defaults and there is no configuration object to learn. It travels as JSON on the graph
node rather than captured in a closure, because one class added twice must yield two
independently configured instances — and because that keeps it visible in `graph`.
> ~~**Configuration is constructor keyword arguments.** `rt.add(Blur, config={"radius": 3})`
> constructs `Blur(radius=3)`, so a processor's settings are ordinary Python parameters with
> ordinary defaults and there is no configuration object to learn.~~ — Superseded 2026-09-11 by
> `agent-readable-processor-catalog.md`: a processor's config is one class, named by the
> annotation on its `__init__`'s `config` parameter, and the helper constructs that class from
> the mapping. Nothing recorded a keyword signature anywhere, so an agent could only learn a key
> by adding the node and reading the failure; a class has annotations and defaults a schema is
> derived from. The rest of the bullet stands: configuration travels as JSON on the graph node
> rather than captured in a closure, because one class added twice must yield two independently
> configured instances — and because that keeps it visible in `graph`.
- **Python ports declare no schema.** The wire is self-describing and consuming is a cast at read
time, so a port carries a name, a description and (on inputs) a delivery profile. Adding a
schema hint here would build on the per-read matching being deleted.
Expand Down
11 changes: 8 additions & 3 deletions docs/plan/changes/agent-readable-processor-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,11 +150,16 @@ from this renders anything new on a port.
is constructed into the class's config class; the `processor` decorator's doc names the
`config` rule; `stubtest` and pyright gate both as today.
- **The six engine-tree fixtures migrate** to a config class in the change; the string
fixture in `test_live_graph_mutation.py` with them. The fourteen example processors and
the four extension-wheel processors lag as §Consumers states
fixture in `test_live_graph_mutation.py` with them. The fourteen example processors
~~and the four extension-wheel processors~~ lag as §Consumers states
(`docs/plan/ARCHITECTURE.md:327-436`: consumers are never in a migration's scope; a
converted consumer's breakage is filed as tracked backlog at that consumer), with the
backlog issues filed at ship naming each file.
backlog issues filed at ship naming each file. — Amended 2026-09-11 by the owner's
ruling that the four extension-wheel processors migrate in the same PR as the engine
half (#2222), as the deliberate canary §Consumers reserves at `:430-433` for in-flight
work: `packages/` is the only consumer tree with a CI lane, so migrating it is what
proves the new construction path on real processors rather than on fixtures alone, and
it keeps that lane green. Only the fourteen example processors owe backlog at ship.

## ADDED: §Processor model — declaration registers

Expand Down
23 changes: 15 additions & 8 deletions packages/streamlib-moq/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,21 +29,28 @@ classifiers = [
# The engine is a binary dependency, never a source one: this wheel links no
# streamlib crate and speaks no engine internals.
#
# 0.18.52 is the earliest release carrying every half this wheel needs. The
# capability-extension mechanism `extension.py:load` is declared against arrived
# in 0.18.48; `read_from_inbound_link_with_timestamp`, which every bag is read
# through, arrived in 0.18.49; and `encode_bag_to_msgpack_bytes`, which every
# data track object is built with, arrived in 0.18.52. Each resolved by `git
# describe --contains` on the commit that introduced it into `_engine.pyi`, not
# from a changelog.
# What this wheel needs arrived across several releases: the
# capability-extension mechanism `extension.py:load` is declared against in
# 0.18.48; `read_from_inbound_link_with_timestamp`, which every bag is read
# through, in 0.18.49; `encode_bag_to_msgpack_bytes`, which every data track
# object is built with, in 0.18.52. Each resolved by `git describe --contains`
# on the commit that introduced it into `_engine.pyi`, not from a changelog.
#
# The binding half is newer than all of them and cannot be named. These
# processors take a config class, which only an engine whose helper constructs
# one can host, and that engine ships in the release *after* this floor was last
# touched — a version the ceiling rule below forbids naming. So 0.20.0 is the
# highest floor that is legal here, not the floor that is true: an install
# against exactly 0.20.0 resolves and then fails in the helper child. Bump this
# to the release carrying config-class hosting once it exists.
#
# The ceiling is the last *published* engine version, not the one in the tree.
# Two lanes install this wheel and they resolve differently: the PR lane
# installs the engine wheel built from this same checkout, while the release
# lane installs from the published index, which is rebuilt only after the
# extension wheels are built. So a floor naming the engine version released in
# the same run would fail the release smoke test.
dependencies = ["streamlib>=0.18.52"]
dependencies = ["streamlib>=0.20.0"]
Comment thread
coderabbitai[bot] marked this conversation as resolved.

# What pip records at install and the engine reads back through
# `importlib.metadata` when a process takes an engine role.
Expand Down
9 changes: 8 additions & 1 deletion packages/streamlib-moq/python/streamlib_moq/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@
"""

from .processors import MoqBroadcastPublisher as MoqBroadcastPublisher
from .processors import MoqBroadcastPublisherConfig as MoqBroadcastPublisherConfig
from .processors import MoqBroadcastSubscriber as MoqBroadcastSubscriber
from .processors import MoqBroadcastSubscriberConfig as MoqBroadcastSubscriberConfig

__all__ = ["MoqBroadcastPublisher", "MoqBroadcastSubscriber"]
__all__ = [
"MoqBroadcastPublisher",
"MoqBroadcastPublisherConfig",
"MoqBroadcastSubscriber",
"MoqBroadcastSubscriberConfig",
]
116 changes: 78 additions & 38 deletions packages/streamlib-moq/python/streamlib_moq/processors.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
import threading
from collections.abc import Mapping, Sequence
from dataclasses import dataclass
from typing import Any, Literal, Protocol
from typing import Annotated, Any, Literal, Protocol

from streamlib import (
EncodedAudioPacket,
Expand Down Expand Up @@ -528,6 +528,27 @@ def _optional_track_names(track_names: Any) -> "list[str] | None":
return names


@dataclass
class MoqBroadcastPublisherConfig:
"""What a `MoqBroadcastPublisher` is configured with."""

relay_url: Annotated[str, "The MoQ relay to publish through."]
broadcast: Annotated[
"str | None", "The namespace to publish under; minted when absent."
] = None
container_format: Annotated[
ContainerFormat, "How each track is packaged on the wire."
] = "cmaf"
delivery_deadline_ms: Annotated[
"int | None",
"How old a bag may be, by its own monotonic stamp, and still be published.",
] = None
track_names: Annotated[
"Sequence[str] | None",
"Names the tracks positionally in wiring order; `streamlib_bag` only.",
] = None


@processor(
description=(
"Publishes encoded video, encoded audio and data bags to a MoQ "
Expand All @@ -542,9 +563,10 @@ class MoqBroadcastPublisher:
for a bag with no `bitstream` key, data. A data track carries any bag at
all, nested whole inside an object beside the publisher's own
`sequence_index` and the bag's stamp, under `streamlib_bag` only. Its
settings are ordinary constructor parameters — `relay_url`, `broadcast`,
settings are a `MoqBroadcastPublisherConfig` — `relay_url`, `broadcast`,
`container_format` and `track_names` — which is what
`rt.add(MoqBroadcastPublisher, config={"relay_url": ...})` passes.
`rt.add(MoqBroadcastPublisher, config={"relay_url": ...})` is constructed
into.

`track_names`, under `streamlib_bag`, names the tracks positionally in
wiring order — the order `runtime.connect` ran — so a subscriber in
Expand Down Expand Up @@ -593,21 +615,16 @@ class MoqBroadcastPublisher:
throughput of about 40 Mbit/s at a 100 ms round trip to the relay.
"""

def __init__(
self,
relay_url: str,
broadcast: "str | None" = None,
container_format: ContainerFormat = "cmaf",
delivery_deadline_ms: "int | None" = None,
track_names: "Sequence[str] | None" = None,
) -> None:
self._relay_url = _required_relay_url(relay_url, "MoqBroadcastPublisher")
self._broadcast = broadcast
def __init__(self, config: MoqBroadcastPublisherConfig) -> None:
self._relay_url = _required_relay_url(config.relay_url, "MoqBroadcastPublisher")
self._broadcast = config.broadcast
self._container_format = _required_container_format(
container_format, "MoqBroadcastPublisher"
config.container_format, "MoqBroadcastPublisher"
)
self._delivery_deadline_ms = _optional_delivery_deadline_ms(
config.delivery_deadline_ms
)
self._delivery_deadline_ms = _optional_delivery_deadline_ms(delivery_deadline_ms)
self._track_names = _optional_track_names(track_names)
self._track_names = _optional_track_names(config.track_names)
self._session: "_native.MoqBroadcastPublishingSession | None" = None
self._kind_by_inbound_link: "dict[str, str]" = {}
self._next_data_sequence_index_by_inbound_link: "dict[str, int]" = {}
Expand Down Expand Up @@ -804,6 +821,26 @@ def _color_axes_of(frame: EncodedVideoFrame) -> "dict[str, str] | None":
return stated or None


@dataclass
class MoqBroadcastSubscriberConfig:
"""What a `MoqBroadcastSubscriber` is configured with."""

relay_url: Annotated[str, "The MoQ relay to subscribe through."]
broadcast: Annotated[str, "The namespace to subscribe to."]
video_track: Annotated[
"str | None", "The track feeding `encoded_video`; unnamed means no video."
] = None
audio_track: Annotated[
"str | None", "The track feeding `encoded_audio`; unnamed means no audio."
] = None
container_format: Annotated[
ContainerFormat, "How each track is packaged on the wire."
] = "cmaf"
data_track: Annotated[
"str | None", "The track feeding `data_bags`; `streamlib_bag` only."
] = None


@processor(
execution="manual",
description=(
Expand Down Expand Up @@ -835,43 +872,46 @@ class MoqBroadcastSubscriber:
would do with the same bytes.
"""

def __init__(
self,
relay_url: str,
broadcast: str,
video_track: "str | None" = None,
audio_track: "str | None" = None,
container_format: ContainerFormat = "cmaf",
data_track: "str | None" = None,
) -> None:
self._relay_url = _required_relay_url(relay_url, "MoqBroadcastSubscriber")
if not isinstance(broadcast, str) or not broadcast:
def __init__(self, config: MoqBroadcastSubscriberConfig) -> None:
self._relay_url = _required_relay_url(
config.relay_url, "MoqBroadcastSubscriber"
)
if not isinstance(config.broadcast, str) or not config.broadcast:
raise ValueError(
"MoqBroadcastSubscriber: `broadcast` is required and names the "
f"namespace to subscribe to; got {broadcast!r}"
f"namespace to subscribe to; got {config.broadcast!r}"
)
if video_track is None and audio_track is None and data_track is None:
if (
config.video_track is None
and config.audio_track is None
and config.data_track is None
):
raise ValueError(
"MoqBroadcastSubscriber: name at least one of `video_track`, "
"`audio_track` and `data_track`; a subscriber naming none would "
"subscribe to nothing and produce nothing."
)
_refuse_track_names_no_broadcast_can_serve(
(("video_track", video_track), ("audio_track", audio_track), ("data_track", data_track))
(
("video_track", config.video_track),
("audio_track", config.audio_track),
("data_track", config.data_track),
)
)
self._container_format = _required_container_format(
container_format, "MoqBroadcastSubscriber"
config.container_format, "MoqBroadcastSubscriber"
)
if data_track is not None and self._container_format == "cmaf":
if config.data_track is not None and self._container_format == "cmaf":
raise ValueError(
f"MoqBroadcastSubscriber: `data_track` names a data track "
f"({data_track!r}), and the `cmaf` container has no packaging for "
f"one; a data track rides `container_format=\"streamlib_bag\"` only."
f"({config.data_track!r}), and the `cmaf` container has no packaging "
"for one; a data track rides "
'`container_format="streamlib_bag"` only.'
)
self._broadcast = broadcast
self._video_track = video_track
self._audio_track = audio_track
self._data_track = data_track
self._broadcast = config.broadcast
self._video_track = config.video_track
self._audio_track = config.audio_track
self._data_track = config.data_track
self._stop = threading.Event()
self._reader: "threading.Thread | None" = None
self._reported_an_oversized_bag = False
Expand Down
18 changes: 14 additions & 4 deletions packages/streamlib-moq/tests/test_data_track_round_trip.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,13 @@
import pytest

from streamlib import ProcessorLinkDataAccess, decode_msgpack_bytes_to_python_object
from streamlib_moq import MoqBroadcastPublisher, MoqBroadcastSubscriber, _native
from streamlib_moq import (
MoqBroadcastPublisher,
MoqBroadcastPublisherConfig,
MoqBroadcastSubscriber,
MoqBroadcastSubscriberConfig,
_native,
)
from streamlib_moq.processors import DATA_BAGS_OUTPUT_PORT, TRACKS_INPUT_PORT

A_RELAY = "https://relay.invalid/a-token"
Expand Down Expand Up @@ -200,14 +206,18 @@ def data_track_round_trip(
)

publishing_session = _ThePublishingSessionKeepingWhatItWasHanded()
publisher = MoqBroadcastPublisher(relay_url=A_RELAY, container_format="streamlib_bag")
publisher = MoqBroadcastPublisher(
MoqBroadcastPublisherConfig(
relay_url=A_RELAY, container_format="streamlib_bag"
)
)
publisher._session = publishing_session # type: ignore[assignment]
subscriber = MoqBroadcastSubscriber(
subscriber = MoqBroadcastSubscriber(MoqBroadcastSubscriberConfig(
relay_url=A_RELAY,
broadcast=A_BROADCAST,
container_format="streamlib_bag",
data_track=THE_DATA_TRACK_NAME,
)
))

yield DataTrackRoundTripUnderTest(
publisher,
Expand Down
Loading
Loading